{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 第一节 线性回归\n",
    "代码修改并注释：廖永,广东工业大学硕士在读,如有错误或指正,请联系136884000@qq.com  \n",
    "代码运行环境：python3.7\n",
    "安装TensorFlow的方法：pip install tensorflow\n",
    "附pandas常用功能函数网址：https://blog.csdn.net/weixin_41770169/article/details/79539232"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt # matplotlib.pyplot是一些命令行风格函数的集合，使matplotlib以类似于MATLAB的方式工作。\n",
    "import seaborn as sns\n",
    "sns.set(context=\"notebook\", style=\"whitegrid\", palette=\"deep\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[1. 0. 0. 0. 0.]\n",
      " [0. 1. 0. 0. 0.]\n",
      " [0. 0. 1. 0. 0.]\n",
      " [0. 0. 0. 1. 0.]\n",
      " [0. 0. 0. 0. 1.]]\n"
     ]
    }
   ],
   "source": [
    "A=np.eye(5)    #单位对角矩阵\n",
    "print(A)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.read_csv('ex1data1.txt',names=['人口','利润'])    # header = None表示没有表头\n",
    "#需要注意的是，Jupyter notebook只能打开当前目录下的数据集，如csv，所以需要使用upload把数据集倒导入到当前目录下。           \n",
    "#pd.read_csv的作用是将csv文件读入并转化为数据框形式，有非常多的参数，用到时可查阅文档。\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>人口</th>\n",
       "      <th>利润</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>6.1101</td>\n",
       "      <td>17.5920</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>5.5277</td>\n",
       "      <td>9.1302</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>8.5186</td>\n",
       "      <td>13.6620</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>7.0032</td>\n",
       "      <td>11.8540</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5.8598</td>\n",
       "      <td>6.8233</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       人口       利润\n",
       "0  6.1101  17.5920\n",
       "1  5.5277   9.1302\n",
       "2  8.5186  13.6620\n",
       "3  7.0032  11.8540\n",
       "4  5.8598   6.8233"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head() #读前5行\n",
    "#括号内可填写要读取的前n行，如果不填，默认为n=5"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 97 entries, 0 to 96\n",
      "Data columns (total 2 columns):\n",
      " #   Column  Non-Null Count  Dtype  \n",
      "---  ------  --------------  -----  \n",
      " 0   人口      97 non-null     float64\n",
      " 1   利润      97 non-null     float64\n",
      "dtypes: float64(2)\n",
      "memory usage: 1.6 KB\n"
     ]
    }
   ],
   "source": [
    "df.info() #查看索引、数据类型和内存信息"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "*****\n",
    "# 通过绘制散点图来观察原始数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "D:\\anaconda3\\lib\\site-packages\\seaborn\\_decorators.py:36: FutureWarning: Pass the following variables as keyword args: x, y. From version 0.12, the only valid positional argument will be `data`, and passing other arguments without an explicit keyword will result in an error or misinterpretation.\n",
      "  warnings.warn(\n",
      "D:\\anaconda3\\lib\\site-packages\\seaborn\\regression.py:580: UserWarning: The `size` parameter has been renamed to `height`; please update your code.\n",
      "  warnings.warn(msg, UserWarning)\n",
      "D:\\anaconda3\\lib\\site-packages\\matplotlib\\backends\\backend_agg.py:238: RuntimeWarning: Glyph 20154 missing from current font.\n",
      "  font.set_text(s, 0.0, flags=flags)\n",
      "D:\\anaconda3\\lib\\site-packages\\matplotlib\\backends\\backend_agg.py:238: RuntimeWarning: Glyph 21475 missing from current font.\n",
      "  font.set_text(s, 0.0, flags=flags)\n",
      "D:\\anaconda3\\lib\\site-packages\\matplotlib\\backends\\backend_agg.py:238: RuntimeWarning: Glyph 21033 missing from current font.\n",
      "  font.set_text(s, 0.0, flags=flags)\n",
      "D:\\anaconda3\\lib\\site-packages\\matplotlib\\backends\\backend_agg.py:238: RuntimeWarning: Glyph 28070 missing from current font.\n",
      "  font.set_text(s, 0.0, flags=flags)\n",
      "D:\\anaconda3\\lib\\site-packages\\matplotlib\\backends\\backend_agg.py:201: RuntimeWarning: Glyph 20154 missing from current font.\n",
      "  font.set_text(s, 0, flags=flags)\n",
      "D:\\anaconda3\\lib\\site-packages\\matplotlib\\backends\\backend_agg.py:201: RuntimeWarning: Glyph 21475 missing from current font.\n",
      "  font.set_text(s, 0, flags=flags)\n",
      "D:\\anaconda3\\lib\\site-packages\\matplotlib\\backends\\backend_agg.py:201: RuntimeWarning: Glyph 21033 missing from current font.\n",
      "  font.set_text(s, 0, flags=flags)\n",
      "D:\\anaconda3\\lib\\site-packages\\matplotlib\\backends\\backend_agg.py:201: RuntimeWarning: Glyph 28070 missing from current font.\n",
      "  font.set_text(s, 0, flags=flags)\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAbIAAAGyCAYAAACBc0EcAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAAuPUlEQVR4nO3df3RU9Z3/8dedmfwmCNEkFKX2+wU8zUpRG3qqtUex/RLREKnRulCr26UeqbVu7bbrEcoe9tgjX75oD62LWs7Wre2BrdDdYJUK6mrhWNhzJNEF001FFKwaO0QCJCEzk/nx+f4xZsjk5ySZOzN37vNxThszk5n7+WSYvObzuZ/7/ljGGCMAABzKk+0GAAAwGQQZAMDRCDIAgKMRZAAARyPIAACO5pggM8YoFAqJRZYAgIEcE2R9fX1qbW1VX1/fpJ7nj3/8Y5palH30JffkSz8k+pKr6MtQjgmydAkGg9luQtrQl9yTL/2Q6Euuoi9DuS7IAAD5xWfnk2/atEm7du2SJF199dW67777tGrVKrW0tKikpESS9J3vfEeLFi2ysxkAgDxmW5Dt379ff/jDH7Rjxw5ZlqU77rhDL774olpbW7VlyxZVVVXZdWgAgIvYNrVYWVmp+++/X4WFhSooKNDs2bPV3t6u9vZ2rV69Wg0NDXrkkUcUi8XsagIAwAWsTBQNPnbsmJYvX66tW7fqxz/+sdauXavy8nKtXLlSS5Ys0S233DLmc4RCIbW2ttrdVABADqqtrR35TmOzw4cPm2uuucY0NTUNue+FF14w3/72t1N6nmAwaJqbm00wGJxUe5qbmyf1+FxCX3JPvvTDGPqSq+jLULauWmxpadE3vvENff/739eNN96oN998U88///zAEJXPZ+t6EwBAnrMtyD788EPdfffdevjhh1VfXy8pHlzr1q3T6dOnFQ6HtW3bNlYsAgAmxbbh0BNPPKFQKKT169cnblu2bJnuvPNOLV++XJFIRHV1dVqyZIldTQAAuIBtQbZmzRqtWbNm2PtuvfVWuw4LAHAZKnsAAByNIAMAOBpLBgEAtmhu86tpzxH5O3tVXVGqxoVztKCmOu3HYUQGAEi75ja/Njcd0smugMpLfDrZFdDmpkNqbvOn/VgEGQAg7Zr2HJHPZ6m40CfLin/1+Sw17TmS9mMRZACAtPN39qqowJt0W1GBV8c7e9N+LIIMAJB21RWlCoWjSbeFwlFVVZSm/VgEGQAg7RoXzlEkYhTsi8iY+NdIxKhx4Zy0H4sgAwCk3YKaaq1snK/pU0vUE4ho+tQSrWycb8uqRZbfAwBssaCm2pbgGowRGQDA0QgyAICjEWQAAEcjyAAAjkaQAQAcjSADADgaQQYAcDSCDADgaAQZAMDRCDIAgKMRZAAARyPIAACORtFgAEiz5ja/mvYckb+zV9UVpWpcOCcjxXPdihEZAKRRc5tfm5sO6WRXQOUlPp3sCmhz0yE1t/mz3bS8RZABQBo17Tkin89ScaFPlhX/6vNZatpzJNtNy1sEGQCkkb+zV0UF3qTbigq8Ot7Zm6UW5T+CDADSqLqiVKFwNOm2UDiqqorSLLUo/xFkAJBGjQvnKBIxCvZFZEz8ayRi1LhwTrablrcIMgBIowU11VrZOF/Tp5aoJxDR9KklWtk4n1WLNmL5PQCk2YKaaoIrgxiRAQAcjSADADgaQQYAcDTOkQGAQ1EKK44RGQA4EKWwziLIAMCBKIV1FkEGAA5EKayzCDIAcCBKYZ1FkAGAA1EK6yyCDAAciFJYZ7H8HgAcilJYcYzIAACORpABAByNIAMAOBrnyADA4dxeqooRGQA4GKWqCDIAcDRKVRFkAOBolKoiyADA0ShVRZABgKNRqoogAwBHo1QVy+8BwPHcXqqKERkAwNEIMgCAoxFkAABHI8gAAI5ma5Bt2rRJ9fX1qq+v14YNGyRJ+/fvV0NDg+rq6rRx40Y7Dw8AcAHbgmz//v36wx/+oB07dujpp5/WH//4R+3cuVOrV6/WY489pueee06tra3au3evXU0AALiAbUFWWVmp+++/X4WFhSooKNDs2bN17NgxXXjhhZo1a5Z8Pp8aGhq0e/duu5oAAHAB24Js7ty5uvTSSyVJx44d065du2RZliorKxM/U1VVJb/fPRWaAQDpZxljjJ0HeOutt7Ry5Urdc8898nq9euWVV/TQQw9Jkvbt26d//dd/1RNPPDHm84RCIbW2ttrZVABAjqqtrR3xPlsre7S0tOjv/u7vtHr1atXX1+vVV19VR0dH4v6Ojg5VVVWN6znnzZunoqKiSbVptF+Ik9CX3JMv/ZDoS66iL0PZNrX44Ycf6u6779bDDz+s+vp6SdIll1yio0eP6t1331U0GtXOnTt11VVX2dUEAIAL2DYie+KJJxQKhbR+/frEbcuWLdP69et1zz33KBQK6eqrr9bixYvtagIAwAVsC7I1a9ZozZo1w973zDPP2HVYAIDLUNkDAOBoBBkAwNEIMgCAoxFkAABHI8gAAI5GkAEAHI0gAwA4GkEGAHA0ggwA4GgEGQDA0QgyAICjEWQAAEcjyAAAjkaQAQAcjSADADgaQQYAcDSCDADgaAQZAMDRCDIAgKP5st0AAEDuaG7zq2nPEfk7e1VdUarGhXO0oKY6280aFSMyAICkeIhtbjqkk10BlZf4dLIroM1Nh9Tc5s9200ZFkAEAJElNe47I57NUXOiTZcW/+nyWmvYcyXbTRkWQAQAkSf7OXhUVeJNuKyrw6nhnb5ZalBrOkU2SE+eTAWA41RWlOtkVUHHh2WgIhaOqqijNYqvGxohsEpw6nwwAw2lcOEeRiFGwLyJj4l8jEaPGhXOy3bRREWST4NT5ZAAYzoKaaq1snK/pU0vUE4ho+tQSrWycn/OzTEwtToK/s1flJcm/QifMJwPASBbUVOd8cA3GiGwSqitKFQpHk25zwnwyAOQTgmwSnDqfDAD5hCCbBKfOJwNAPuEc2SQ5cT4ZgL24LCezGJEBQBpxWU7mEWQAkEZclpN5BBkApJFTyzw5GUEGAGnEZTmZR5ABQBpxWU7mEWQAkEZclpN5LL8HgDTjspzMYkQGAHA0ggwA4GgEGQDA0QgyAICjEWQAAEcjyAAAjkaQAQAcjSADADgaQQYAcDSCDADgaAQZAMDRCDIAgKNRNBjII81tfjXtOSJ/Z6+qK0rVuHAOxWuR9xiRAXmiuc2vzU2HdLIroPISn052BbS56ZCa2/zZbhpgK4IMyBNNe47I57NUXOiTZcW/+nyWmvYcyXbTAFsRZECe8Hf2qqjAm3RbUYFXxzt7s9QiIDMIMiBPVFeUKhSOJt0WCkdVVVGapRYBmUGQAXmiceEcRSJGwb6IjIl/jUSMGhfOyXbTAFsRZECeWFBTrZWN8zV9aol6AhFNn1qilY3zWbWIvGfr8vuenh4tW7ZMP/vZz3TBBRdo1apVamlpUUlJiSTpO9/5jhYtWmRnEwBXWVBTTXDBdWwLsoMHD2rNmjU6duxY4rbW1lZt2bJFVVVVdh0WAOAytk0tbt++XWvXrk2EViAQUHt7u1avXq2GhgY98sgjisVidh0eAOASljHG2HmAL33pS/rVr34lY4zWr1+vtWvXqry8XCtXrtSSJUt0yy23pPQ8oVBIra2tdjYVAJCjamtrR7wvYyWqZs2apUcffTTx/W233aann3465SDrN2/ePBUVFU24HS0tLaP+QpyEvuSe8fYjl0tK5ctrItGXXJWuvmRs1eKbb76p559/PvG9MUY+H6Ue4V6UlALSI2NBZozRunXrdPr0aYXDYW3bto0Vi3A1SkoB6ZGxIdGnP/1p3XnnnVq+fLkikYjq6uq0ZMmSTB0eyDn+zl6VlyS/BSkpBYyf7UH28ssvJ/771ltv1a233mr3IQFHqK4o1cmugIoLz74NKSkFjB+VPYAsoaQUkB4EGZAllJQC0oNlg0AWUVIKmDxGZAAARyPIAACORpABAByNIAMAOBpBBgBwNIIMAOBoLL9HTldgB4CxMCJzOSqwA3A6RmQuN7ACuyQVF/oUVERNe44wKksTRryAvRiRuZy/s1dFBd6k26jAnj6MeAH7EWQuV11RqlA4mnQbFdjThz3HAPsRZC5HBXZ7MeIF7EeQuRwV2O3FiBewH4s9QAV2GzUunKPNTYcUVERFBV6FwlFGvECaMSIDbMSIF7AfIzLAZox4AXsxIgMAOBpBBgBwNIIMAOBoBBkAwNEIMgCAoxFkAABHI8gAAI5GkAEAHI0gAwA4GkEGAHA0SlRhUtj9GEC2EWRIGG8o9e9+7PNZSbsfi6K4jjT49f/MBUa1tdluFTA2phYh6WwonewKJIVSc5t/xMew+3H+GO7133Xg1KivP5ArGJHlkExP0w083plAWMWFXk0pLZIkFRf6FFRETXuOjNgGf2evykuS/wmx+7EzDfxQIsVf/0BQo77+QK5gRJYjJjIiSufxgqGITnWHdCYYSfzMWKHE7sf5w9/Zq6ICb9JtBV6LDyVwBIIsR2R6mm7w8QoLPJIlneoOJX5mrFBqXDhHkYhRsC8iY+Jf2f3YmYb7UBKOGj6UwBEIshwx3CdiO6fpBh9v2pQiWZL6wtGUQ4ndj/PHcB9KYlHxoQSOwDmyHFFdUaqTXYHEOQrJ3mm6wccrKylQXySqYF9MPYGIqlI8R8fux/lhQU211DhfTXuO6Hhnr6oqSnXVXxXz2sIRCLIc0bhwjjY3HVJQERUVeBUKR22dphvueAVer75z62X88XKpwR9KWlpastgaIHVMLeaITE/TMS0IIF8wIsshmZ6mY1oQQD5gRAYAcDSCDADgaAQZAMDRCDIAgKMRZAAARxtz1eIrr7yiysrKYe975513dP3116e9UQAApGrMIHvttdf03e9+d9j7nn/++bQ3CACA8RhzatEYM6H7AADIhDGDzLKsCd0HAEAmsNgDAOBoTC0CABxtzMUen/3sZ/WnP/1p2Pvmzp2b9gYBADAeYwbZVVddNeJ9n/70p9PaGAAAxovryAAAjsZ1ZAAARxszyFjsAUxOc5tfTXuOyN/Zq+qKUjUunMM+cEAa2XodWU9Pj5YsWaL3339fkrR//341NDSorq5OGzduHGdTAedpbvNrc9MhnewKqLzEp5NdAW1uOqTmNn+2mwbkDduuIzt48KCWL1+uY8eOSZKCwaBWr16txx57TM8995xaW1u1d+9euw4P5ISmPUfk81kqLvTJsuJffT5LTXuOZLtpQN6w7Tqy7du3a+3ataqqqpIkHTp0SBdeeKFmzZoln8+nhoYG7d69ewJNBpzD39mrogJv0m1FBV4d7+zNUouA/GPbdWQPPvhg0vfHjx9PWv1YVVUlv5/pFeS36opSnewKqLjw7FstFI6qqqI0i60C8kvGriOLxWJJ59SMMROq1dja2jruxwzW0tIy6efIFfQl9wzsx2cuMNp1IKhAUCrwWgpHjWJR6aq/KnZEf53QxlTRl9yUal9qa2tHvG/MIEuXGTNmqKOjI/F9R0dHYtpxPObNm6eioqIJt6OlpWXUX4iT0JfcM7gftbXS3DnxVYvHO3s1s8o5qxbz5TWR6EuuSldfMhZkl1xyiY4ePap3331XF1xwgXbu3KmbbropU4cHsmZBTbUjggtwqowFWVFRkdavX6977rlHoVBIV199tRYvXpypwwMA8pTtQfbyyy8n/vuKK67QM888Y/chAQAuwn5kAABHI8gAAI5GkAEAHC1jiz2chCKvAOAcBNkg/UVefT4rqcirGucTZsgIPkgB48PU4iAUeUU2US0fGD+CbBCKvCKb+CAFjB9BNkh1RalC4WjSbRR5RabwQQoYP4JskMaFcxSJGAX7IjIm/jUSMWpcOCfbTYML8EEKGD+CbJAFNdVa2Thf06eWqCcQ0fSpJVrJQg9kCB+kgPFj1eIwKPKKbFlQUy01zk9Uy69i1SIwJoIMyDF8kALGh6lFAICjEWQAAEcjyAAAjkaQAQAcjSADADgaQQYAcDSW3wOi4jzgZIzI4HpUnAecjSCD61FxHnA2phaRU7Ixxefv7FV5SfJbgYrzgHMwIkPOyNYUHxXnAWcjyJAzsjXFR8V5wNmYWkTGjDVtmK0pPirOA85GkCEj+qcNfT4radpQA/Z6q64o1cmugIoLz/6zzNQUHxXnAediahEZkcq0IVN8ACaCIENG+Dt7VVTgTbpt8LQhu3MDmAimFscpnypAZLIvqU4bMsUHYLwYkY1DPlWAyHRfmDYEYBdGZOMw8DyPJBUX+hRURE17jjhuFJHpvmRiZWA+jZYBpI4gG4d8qgCRjb7YOW2YyqpIAPmJqcVxyKcKEPnUF4l6iYCbEWTj8JnZ58rfGdDR9i69f7xHJ7tDjj3Pk2/nrFJZFQkgPzG1mKLmNr9ePvCeppYV6EwgonAkpu4zfbr5y3OzMnXV3ObXL1/q0GO7X0ycD5KU8jmidJyzsuuc1ESeN5sXUwPILoIsRf1TV1NKizW9PH5bsC+iN94+oeUZbkv/+aBwJKJp5UU62RXQI9telzFGU0oLUj5HNJlzVuk+J3W4PaD/eHWf/vyXLvUGI5paVqhpUwpTft7GhXO0uemQgoqoqMCrUDjq6BEmgNQxtZiiiU5dNbf5tfrxffrmgy9q9eP70rK8vT9UC32exPmg3mBEgVAkY+eI0nlOqrnNr10HTulkV0Chvqhixuh0T596Q9GUn5eLqQH3YkSWoolMXdm1kq5/xWFv5OxtkWhMlpX8c4ODNp1Tgelc9di054g83vglAJFoTF7LkpF0qjuksmJfys/LxdSAOzEiS9FEFkfYtZJuuBWHPq9HXk9ykg0M2nRfAJ3OVY/+zl4VeONtL/B5ZIzksaRwJDap5wXgDgRZiiYydWXXSrr+UO2LxBKhWlrsU0mRb8SgTXeopnPVY3VFqcJRI0maNqVIRkaRmJHPazl+NSUA+zG1OA7jnbqyayVd/4rDXz77unoCEVVVlOqbN5wNrOFWIab7Auh0VupoXDhHj/z6QCKQz5lSqO4zYRUX+jR9agkVOgCMiiCzkZ0r6RbUVMvqrVRtbe2Q24djR6im65zUgppqXfe5aXrjfUvHO3s1s7JcjV8lvACkhiCzUS7tPJzry9Mvmlmi5Q21Y/8gAAxCkNksV1bS5VKoAkA6EWQZkguV2XMlVAEgnVi1mAH5tI8ZAOQagiwDqMwOAPZhajEDcm0fs1yY5gSAdHHViKy5za8nX+pIa93DVOTS3l9McwLIN64Jsv4/4N2BSMb/gOfS3l9McwLIN64JsuEqxmfqD3guVWZnA0oA+cY158iGqxhv1x/wkc5B5cJ5KDagBJBvXBNk/X/AB7LjD/hkt26xeyGG3RU+nLqQxKntBuCiqcXhKsbbcZ6qfwozGpPaP+rVX04E9NGpgP7frw6MucgkEwsx7JzmdOpCEqe2G0Cca0Zkw1WMt+NTt7+zVx5L+uhUML7RpYkpEpMifVFNszTqCG3gQgwpvtFkUBE17TmS1nbaNc2Zqfanm1PbDSDONUE2kLHxuasrStV29ISiMSMz4ECWJZ3u6dP5lWUj/pHMtevNpPFNueVi+1Ph1HYDiHPN1GKmlt9/Zva5ikSTQ0ySPJaV2PF4pD+SuXS9mTT+Kbdca3+qnNpuAHFZCbLbbrtN9fX1Wrp0qZYuXaqDBw/afsxMLb9/4+0T8nms+LTiADFjVOCL/7pH+iM53uvNDrcHtPrxfbZd4D3ea85y6Xq58XBquwHEZXxq0RijY8eO6fe//718vswdPlPL7/2dvTpvekniHJmJxc+RGSOdM6Vw1D+S49lqpbnNr10HTqmsrHhCqyNT7ct4ptyculWMU9sNIC7jQfbOO+9IklasWKFTp07plltu0de//nXbj5up5ff9x6mcXqJT3SGFjeTzGPl8HhkjTZ9aMuofyYELMfrPTz3edGjI+ammPUfk8crWBQoTueYsV66XGy+nthuAZBkz+GyOvV5//XX9+te/1j/+4z8qHA7r9ttv16pVq3TllVeO+rhQKKTW1tYJH/dwe0C7DpySxysVeC2Fo0axqHTd56bpopklE35eu44z1vP85LcfqrjQkjVgDjM+LWZ079JP5FRfAGCyamtH3kE+4yOyyy67TJdddlni+5tvvll79+4dM8j6zZs3T0VFReM+bm2tNHeOX7989nUFwl7NrLJn+qj/OP3TVBM9zn+8uk9lZcVJo6FgX0RvvG9peUOtZr26T+3HT2r61LKk+2edWzLqC56NvqSipaUlbe3Opnzph0RfchV9GSrjQdbc3KxwOKwrrrhCUnwUkalzZQtqqmX1Vqb8i5totYd0TFONdX6qceEcPfLrAwr22VOhox9TbgByXcZXLXZ3d2vDhg0KhULq6enRjh07tGjRokw3Y0zZrvYw1pLwBTXVuu5z00as0NHc5rd1RSMA5IqMj8iuueYaHTx4UF/5ylcUi8X0ta99LWmqMVdku9pDKjURL5pZouUNQ0eXk633CABOkpXKHvfee6/uvffebBw6Zdmu9jCZJeHZDmEAyCRXlqhKRS5sdzL4/FRzm1/3PPx7fdDRI0maXubR/znxJ73x9omk83jZDmEAyCSCbJD+BR5//kuXeoMRTS0r1LQphbYtphhPux7Z9rq6zvTJsiRL0vHTET31wpuaPrVY06YUJqYQS4riU5GphjBbmABwMtfUWkzFwAUe551TrKllBeo+06cTp0NZ3dVZik8X9gYj8ngseT0eeTzxly5mpN5gJKmElCUr5ZJL2V7UAgCTRZANMLi24PTyYlVVlGjWjHKtu+vKrI5S/J29ikRj8lhD7+svRizFpxADoUjSnmNer0fFRT493nRoyArG8dZTBIBcQ5AN4O/sVVGBN+m2XDi31Nzm15lAWNFYfGPQaDSWdH9/MWLp7BTigppqrbvrSn2rcb5Coagi0eiwI65c7TMApIpzZANkY4HHWOen+qf+igu9CgTDihkpYoxiJn6NmceSSot9MsYMex5vrBWM6eoz59kAZAsjsgEyvZ1HKuen+oNoenmRqs8tU2HB2XNjBd54QAX7oiOexxtrxJWOPnOeDUA2uSrImtv8evKljhGrXSyoqU46t2T3Ao9Uzk8NDKKyYp8+WV2uGRUl8nosTS3z6BPnlWp6eaGKC73DjoJSqRAy2T5zng1ANrlmarF/1BCORDStvGjEaheZrC2YyvVew039newOyef1JG0SOtIFz6lUCJlsn7luDUA2uWZElqkdogcaq97hWKMlafipv3DUaPrU5B0ARgqOTIwyU+kHANjFNSOyTO0Q3S+VeoepjpYGl6ryeb2KRKOKDujLaMFh9ygzlX4AgF1cE2Rj7RCd7lV3qdQ7TLWe4nClquLTpDGVjrBaMZMmUxcSACbLNUHWP2oY7o//cKOnnz71mqaVF6s3FJlQsKV63mgio6X+4Pjls6+rJxDJieBg3zIA2eKaIBvtj//qx/cljZ6iUaPu3rACoaguqCqb0DYo6bg+a7RR4ng3CQWAfOWaIJNG/uM/ePR0qickjyVFY0a9oahOdYfUF47q4a0t+sGttSmF2WTPG7GnGACkxjWrFkczeNVdOBKTkeTxSB0nA+qLRBWNGZ0JhPV/n3xVv37hT0OeY/AKRUmTWi3ItVkAkBpXjchGMnj0JEmRqJGiJunnLEuKGqPfvPSW5s6angilkUZPKxvna91dV06oTVybBQCpYUSm5GutPjodVMyYYavMey1LPo+lWMwkjYzsGD1xbRYApIYg+1h/tfhPzpiqmeeVqbqiVB4rOc28Xk+8xqHPkzQysqOCfKbrPgKAUxFkg/SHUllJgarPLdXAKIsZI2OkspKCpJGRHaOnTNd9BACnctU5suY2v375Uoce2/3iiNeGDVw2X1bs0/SpRersCkmSvB5LZSUFKvB6kkZGdlW24NosABiba0Zk/QsyugORUbcaGTylV1rs0/TyIl34iXKdM6VIMyunDBkZMXoCgOxxzYisf0GGpdErxg9XbumbN4xdNYPREwBkh2uCbDxFgwklAHAO1wTZWEWD0yXdxYcBAKNzTZCNVjQ4FakEFGWlACDzXLPYo39BRnmJb9wLMvoD6mRXYNSFIpSVAoDMc82ITJp4xfhU9haTKCsFANngmhHZZKRauYOyUgCQeQRZClINKMpKAUDmuWpqUZIOtwf0H6/uG9eqwlQrdwx3DRqrFgHAXq4KsuY2v3YdOKWysuLEoo2fPvWappUXqzcUGTHYxhNQXIMGAJnlqiBr2nNEHq8SizaiUaPu3rACoaguqCobdrl8/7L79/7SrXA0JhMz6gmE9ZOnXtMnZ0xlxAUAWeaqc2T+zl4VeM/Wsz/VEy8G3BeO6t2/9OjE6ZDC0VhiuXz/svv2jh519/YpEAqrNxRRbzCsM4Gw2ju6h12GDwDIHFcFWXVFqcIDdn3uC8cUjRnJkrweKRKN6VR3SO/9pVvS2WX3vcGIPB5LZsCG0Zbit3OdGABkl6uCrHHhHMWiSqwqjH2cTD6PR5IV30jTksLRmKSzy+7DkZg8lhJBZoxkWVI4EuM6MQDIMlcF2YKaal33uWmJ7Vasj0dikuLBFjOyJPk+nn7sX3Zf4IvvDN2/YbT1cagV+DxcJwYAWeaqIJOki2aWaN1dV+rnP1yki//3eZpWXiSfz1LMGPl8ls6ZUqhPzpgq6ex1YaXFvnjIDdgu2ih+O9eJAUB2uWrV4mD914ede45v2OvDBi67j0ZNYtWi5bHk81qaWVk+YvFgKuADQGa4OshSuT5svNeFUQEfADLL1UEmTfwC5pFGXakWGAYApIfrg2wiRht1UQEfADLLdYs90mG0fceogA8AmcWIbJDmNr9++bv/0QcdPZKkmZVl+kb9xSnvO/atxvkpFRge7fgsFAGA1DEiG6C5za9Htr2u9/zdMsZIxuh9f49++tRrSWWoqitKdaonpA86enTswy590NGjUz0hVVWUJnai7r9WzY6dqAEAZzEiG6Bpz5FEOSpP/0VjMaNAKHmxxmdmn6v/eedEvLSVJfVFYurr7tO1l58raeILSFgoAgDjx4hsAH9nryLReDmqfpYlRWMmabHGG2+f0LTyIhX6vIoZS4U+r6aVF+mNt09M+vip7EQNADiLEdkA1RWlOt0dUswYeSwpZozCkXjdxZ5AWM1tfi2oqZa/s1fTphRqenlR4rHGmEkHTnVFqU52BRIjMomFIgAwFkZkAzQunJMoRxWORhMh5rGk4kJP4nzVWCsTm9v8Wv34Pn3zwRe1+vF9KZ/j6i+J1V/UONgXoQQWAIzBlSOy4VYGStKTv/ujunr74lu7fKzAZ+m8c0pUVlKgYF/8fFV/aavhViZOprLHeHaiBgDEuS7IDrcH9PKh5KD56VOvKRyJKdgXlceSLK+lSNTI65HOO6dEkvRBR4/CkZj8nQFJ0soRAmf14/smtWBjogtFAMCtXBdk+9t65PP5koKm42RAfZGYfB6PPB+v9IjGoorGpBOng4oZI0vx2z2Wpc1Nh7Sycb7W3XXlkOensgcAZJbrzpGd6okMWRkYjZnEZpn9vJ74r6YvEotH2Mf/V3FO8ai7QlPZAwAyy3VBNm2Kb0jQeD1WYrPMfpYV32DTsqSYkXxejyqnl6is2DfqCGu4BRtnAhF1nQmNe/EHAGBsrguyL9RMUSRidLI7qPeP9+hoe5eiMaNCn0cxYxSLxRSNxRSLGZWXFuqTM8r1ifNKdX5lmcqK41OGo42wBlf28Hm9MsYoGo1RrQMAbOC6c2QXzSyRis7Tb156S7GYUYHPo7KSAsVi8R2iO7tCisWMvB5LU6cU6sr5M/XygffGVTtx4IKN1Y/vUyQazalqHdRzBJBPsjIie/bZZ3X99derrq5OW7duzfjx33j7hKorSvS/Zk7VBVVTNL28SGUlPpUWF6jynBKdX1mmC2eUKxqN6eUD7+lLn5s1odqJUu5V66CeI4B8k/ERmd/v18aNG9XU1KTCwkItW7ZMn//85zVnTuYu+h1pZeGf/T2qrigZMnp64+0Tw65QTEWuVeugniOAfJPxEdn+/ft1+eWXa9q0aSotLdW1116r3bt3Z7QNI60slJT20VOuVevItREiAExWxoPs+PHjqqysTHxfVVUlvz+z01ojhcv5lVPSvnR+Mtu62IHLAwDkG8uYgYvO7ff4448rFArp3nvvlSRt375dra2teuCBB0Z9XCgUUmtra9racbg9oP1tPTrVE9G0KT59oWaKJGnXgVPyeKUCr6Vw1CgWla773LT4IpE8cLg9kPd9BJB/amtrR7wv4+fIZsyYoebm5sT3HR0dqqqqSvnx8+bNU1FR0dg/OIKWlhbV1taqtlZa3jD0/rlz/InSUzOrcntFX39fxqO2Njf7OJG+5KJ86YdEX3IVfRkq40H2hS98Qf/8z/+szs5OlZSU6IUXXtCPfvSjTDdjRG6odeiGPgJwj4wHWXV1tb73ve/p9ttvVzgc1s0336z58+dnuhkAgDyRlQuiGxoa1NAwzLweAADj5LoSVQCA/OK6ElX9KNMEAPnBlSMyyjQBQP5wZZANLNNkWfGvo+0xBgDIXa4MMso0AUD+cGWQUaYJAPKHK4Ms1wr5AgAmzpVBlmuFfAEAE+fa5ff9odW/BL9/oQdhBgDO4soRmcQSfADIF64dkdm9UzIXXANAZrh2RGbnEnxGewCQOa4NMjuX4HPBNQBkjmuDzM4l+FxwDQCZ49ogs3MJPhdcA0DmuHaxh2TfTsmNC+doc9MhBRVRUYFXoXCUC64BwCauHZHZiQuuASBzXD0is5Ndoz0AQDJGZAAARyPIAACORpABAByNIAMAOBpBBgBwNIIMAOBoBBkAwNEIMgCAoxFkAABHc01lj/6NLt/78KRmvbqPjS4BIE+4YkQ2cKPL4kKLjS4BII+4IsjY6BIA8pcrgoyNLgEgf7kiyNjoEgDylyuCrHHhHEUiRsG+iIyJf2WjSwDID64IsoEbXQb7DBtdAkAecc3y+/6NLltaWlRbW5vt5gAA0sQVIzIAQP4iyAAAjkaQAQAcjSADADgaQQYAcDSCDADgaAQZAMDRCDIAgKMRZAAARyPIAACORpABABzNMbUWjTGSpL6+vkk/VygUmvRz5Ar6knvypR8SfclVbu1LYWGhLMsacrtl+hMix3V3d+vw4cPZbgYAIEvmzZunoqKiIbc7JshisZjOnDmjgoKCYRMZAJDfHD8iAwBgOCz2AAA4GkEGAHA0ggwA4GgEGQDA0QgyAICjEWQAAEcjyAAAjpa3QXbbbbepvr5eS5cu1dKlS3Xw4MGk+9va2tTY2Khrr71WP/zhDxWJRLLU0tH95je/SfRh6dKlqq2t1QMPPJD0M5s2bdI111yT+JmtW7dmqbXD6+np0ZIlS/T+++9Lkvbv36+GhgbV1dVp48aNwz6mvb1dt956qxYvXqy77rpLZ86cyWSTRzS4L9u2bdOSJUvU0NCgVatWDVtCbceOHfriF7+YeH1G6nOmDe7LqlWrVFdXl2jniy++OOQxTnhd9u7dm/Seufzyy7Vy5cohj8nF12XTpk2qr69XfX29NmzYIMmZ75fh+mHre8XkoVgsZr74xS+acDg84s/U19eb119/3RhjzKpVq8zWrVsz1LqJO3z4sFm0aJE5ceJE0u0rV640r732WpZaNbr//u//NkuWLDEXX3yxee+990wgEDBXX321+fOf/2zC4bBZsWKF2bNnz5DH3XnnnWbnzp3GGGM2bdpkNmzYkOmmDzG4L++8845ZtGiR6e7uNrFYzNx3333mF7/4xZDHPfDAA+bZZ5/NfINHMbgvxhizZMkS4/f7R32cE16XgY4fP26+/OUvm6NHjw55XK69Lvv27TN//dd/bUKhkOnr6zO33367efbZZx33fhmuH5s3b7b1vZKXI7J33nlHkrRixQrdcMMN2rJlS9L9H3zwgYLBoC699FJJUmNjo3bv3p3pZo7bP/3TP+l73/ueKioqkm5vbW3V5s2b1dDQoAceeCCnCopu375da9euVVVVlSTp0KFDuvDCCzVr1iz5fD41NDQM+d2Hw2EdOHBA1157raTceX0G96WwsFBr167VlClTZFmWLrroIrW3tw953BtvvKEdO3aooaFBP/jBD3T69OlMN32IwX0JBAJqb2/X6tWr1dDQoEceeUSxWCzpMU55XQbasGGDli1bpk996lND7su116WyslL333+/CgsLVVBQoNmzZ+vYsWOOe78M14++vj5b3yt5GWRdXV264oor9Oijj+rJJ5/UU089pX379iXuP378uCorKxPfV1ZWyu/3Z6OpKdu/f7+CwaCuu+66pNvPnDmjmpoa/cM//IN27Nihrq4uPfbYY1lq5VAPPvigFixYkPh+8O++qqpqyO/+5MmTmjJliny++OYMufL6DO7L+eefryuvvFKS1NnZqa1bt+rLX/7ykMdVVlbq29/+tp555hl94hOfGDI1nA2D+/LRRx/p8ssv17p167R9+3Y1Nzfr3//935Me45TXpd+xY8f06quv6vbbbx/2cbn2usydOzfx4frYsWPatWuXLMty3PtluH4sWbLE1vdKXgbZZZddpg0bNqi8vFwVFRW6+eabtXfv3sT9sVgsqfCkMSbnCxE/9dRT+tu//dsht5eVlelf/uVfNHv2bPl8Pq1YsSKpr7kmld/9cLfl8uvj9/v1N3/zN7rpppv0+c9/fsj9jz76qGpra2VZlu644w698sorWWjl6GbNmqVHH31UVVVVKikp0W233Tbk35HTXpdt27bpa1/7mgoLC4e9P1dfl7feeksrVqzQfffdp1mzZjn2/TKwH/0jYrveK3kZZM3Nzfqv//qvxPfGmMSnFUmaMWOGOjo6Et9/9NFHw05L5Iq+vj4dOHBAX/rSl4bc197envTJeXBfc83g331HR8eQ331FRYW6u7sVjUZH/Jlc8fbbb2vZsmW68cYbdffddw+5v7u7W08++WTie2OMvF5vBluYmjfffFPPP/984vvh/h056XWRpJdeeknXX3/9sPfl6uvS0tKib3zjG/r+97+vG2+80bHvl8H9kOx9r+RlkHV3d2vDhg0KhULq6enRjh07tGjRosT9559/voqKitTS0iJJ+u1vf6urrroqW80d05tvvqlPfepTKi0tHXJfcXGxHnroIb333nsyxmjr1q1Jfc01l1xyiY4ePap3331X0WhUO3fuHPK7Lygo0IIFC/Tcc89Jkp5++umcfH16enr0zW9+U9/97ne1YsWKYX+mtLRUP//5zxOrZrds2ZKTr48xRuvWrdPp06cVDoe1bdu2Ie10yusixaevgsGgZs2aNez9ufi6fPjhh7r77rv18MMPq76+XpIz3y/D9cP298qElqU4wMaNG83ixYtNXV2defLJJ40xxtxxxx3m0KFDxhhj2trazE033WSuvfZa8/d///cmFApls7mj+t3vfmfuvffepNsG9mX37t2mvr7e1NXVmfvvvz8n+3LNNdckVpTt37/fNDQ0mLq6OvPggw+aWCxmjDFm9erV5j//8z+NMca8//775utf/7q57rrrzIoVK8ypU6ey1vbB+vvyi1/8wlx88cXmhhtuSPzvJz/5iTEmuS8HDhwwX/nKV8zixYvNt771LdPV1ZXN5icZ+Lps2bLFXHfddWbRokXmoYceSvyM014XY4w5ePCg+epXvzrkZ3L5dfnRj35kLr300qR/T//2b//muPfLcP342c9+Zut7hf3IAACOlpdTiwAA9yDIAACORpABAByNIAMAOBpBBgBwtNy9chZwob179+q1114bcvtnP/tZXX311WPeD7gRy++BHNLW1qaampoRbx/rfsCNmFoEADgaQQYAcDSCDADgaAQZAMDRCDIAgKOx/B7IIW+//faw29TPnTtXNTU1Y94PuBHL7wEAjsbUIgDA0QgyAICjEWQAAEcjyAAAjkaQAQAcjSADADgaQQYAcDSCDADgaP8fY/ZScBRTXP4AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x432 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.lmplot('人口','利润',data=df,size=6,fit_reg = False)\n",
    "#fit_reg:拟合回归参数,如果fit_reg=True则散点图中则出现拟合直线\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***\n",
    "# 计算代价函数"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "首先，我们将创建一个以参数θ为特征函数的代价函数\n",
    "$$J\\left( \\theta  \\right)=\\frac{1}{2m}\\sum\\limits_{i=1}^{m}{{{\\left( {{h}_{\\theta }}\\left( {{x}^{(i)}} \\right)-{{y}^{(i)}} \\right)}^{2}}}$$\n",
    "其中：\\\\[{{h}_{\\theta }}\\left( x \\right)={{\\theta }^{T}}X={{\\theta }_{0}}{{x}_{0}}+{{\\theta }_{1}}{{x}_{1}}+{{\\theta }_{2}}{{x}_{2}}+...+{{\\theta }_{n}}{{x}_{n}}\\\\] "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "def computeCost (X,y,theta):\n",
    "    inner=np.power((X*theta.T)-y,2)  \n",
    "    #theta.T就是矩阵theta的转置矩阵\n",
    "    #np.power(A,B)   ## 对A中的每个元素求B次方\n",
    "    return np.sum(inner)/(2*len(X))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "让我们在数据中添加了一列，以便我们可以使用向量化的解决方案来计算代价和梯度。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "df.insert(0,'ONE',1)  #在第0列插入表头为“ONE”的列，数值为1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "下面对变量进行初始化"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "#设置训练值变量X和目标变量y\n",
    "cols=df.shape[1] #获取表格df的列数\n",
    "X=df.iloc[:,0:cols-1] #除最后一列外，取其他列的所有行，即X为O和人口组成的列表\n",
    "y=df.iloc[:,cols-1:cols]#取最后一列的所有行，即y为利润\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "观察一下X和y是否正确"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ONE</th>\n",
       "      <th>人口</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>6.1101</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>5.5277</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>8.5186</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>7.0032</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>5.8598</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   ONE      人口\n",
       "0    1  6.1101\n",
       "1    1  5.5277\n",
       "2    1  8.5186\n",
       "3    1  7.0032\n",
       "4    1  5.8598"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X.head()#看X的前5行"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>利润</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>17.5920</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>9.1302</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>13.6620</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>11.8540</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>6.8233</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        利润\n",
       "0  17.5920\n",
       "1   9.1302\n",
       "2  13.6620\n",
       "3  11.8540\n",
       "4   6.8233"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y.head()#看y的前5行"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "代价函数是应该是numpy矩阵，所以我们需要转换X和Y，然后才能使用它们。 我们还需要初始化theta。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "X = np.matrix(X.values)\n",
    "y = np.matrix(y.values)\n",
    "theta = np.matrix(np.array([0,0]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "matrix([[0, 0]])"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "theta"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((97, 2), (97, 1), (1, 2))"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X.shape,y.shape,theta.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "计算代价函数 (theta初始值为0)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "32.072733877455676"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "computeCost (X,y,theta)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# batch gradient decent（批量梯度下降）对应笔记P21页\n",
    "$${{\\theta }_{j}}:={{\\theta }_{j}}-\\alpha \\frac{\\partial }{\\partial {{\\theta }_{j}}}J\\left( \\theta  \\right)$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "def gradientDescent(X,y,theta,alpha,iters):      #alpha是学习率,iters为迭代次数\n",
    "    temp=np.matrix(np.zeros(theta.shape)) #np.zeros(theta.shape)=[0.,0.],然后将temp变为矩阵[0.,0.]\n",
    "    parameters= int(theta.ravel().shape[1])  \n",
    "    #theta.ravel()：将多维数组theta降为一维，.shape[1]是统计这个一维数组有多少个元\n",
    "    #parameters表示参数\n",
    "    cost=np.zeros(iters)     #初始化代价函数值为0数组，元素个数为迭代次数\n",
    "    \n",
    "    for i in range(iters):   #循环iters次\n",
    "        error=(X*theta.T)-y\n",
    "        \n",
    "        \n",
    "        for j in range(parameters):\n",
    "            term = np.multiply(error, X[:,j])  #将误差与训练数据相乘，term为偏导数，参考笔记P27\n",
    "            temp[0,j] = theta[0,j] - ((alpha / len(X)) * np.sum(term)) #更新theta\n",
    "        \n",
    "        \n",
    "        theta=temp\n",
    "        cost[i] = computeCost(X,y,theta)  #计算每一次的代价函数\n",
    "           \n",
    "    return theta,cost\n",
    "\n",
    "        \n",
    "        \n",
    "        \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "初始化一些附加变量,比如学习速率α和要执行的迭代次数。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "alpha=0.01\n",
    "iters=1500"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "现在让我们运行梯度下降算法来将我们的参数θ适合于训练集。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "g, cost = gradientDescent(X, y, theta, alpha, iters) #令g和cost分别等于函数的两个返回值"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "scrolled": true
   },
   "source": [
    "看一下g"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "matrix([[-3.63029144,  1.16636235]])"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "g"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "看一下代价函数cost"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([6.73719046, 5.93159357, 5.90115471, ..., 4.48343473, 4.48341145,\n",
       "       4.48338826])"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cost"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "最后，我们可以使用我们拟合的参数计算训练模型的代价函数（误差）。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "4.483388256587726"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "computeCost(X, y, g)  #最小化的低价函数"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "现在我们来绘制线性模型以及数据，直观地看出它的拟合。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAs8AAAHwCAYAAABZtoJSAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAABpk0lEQVR4nO3deXhTZfo+8DtJ032FbkBL2WkR2SqyyI4shUKBFhSVqijggigqjCheqKMOP2V0dGCGccavMyiKbbqylEWgLLIHKFTLolBIuhdK96Zpcn5/MO1QuqVtkpOk9+e65hpzmpzz5G2sd948eV+JIAgCiIiIiIioRVKxCyAiIiIishYMz0REREREBmJ4JiIiIiIyEMMzEREREZGBGJ6JiIiIiAzE8ExEREREZCCGZyIyG7VajZCQEERERNT9b/bs2VAoFO0+97JlyxAfHw8AiIiIQElJSZP3LS0tRXR0dKuvsXv3bixatKjBcWM+r4yMDDz66KOYN28etmzZgg8//BAAkJqaii+++KLV52sNU/5+GhMfH49ly5a1eL+1a9ciPT0dAPDOO+/g2LFjRq1h3rx5mD17NmbOnIl33nkHpaWlAIAffvgBX331ldGuRUS2wU7sAoioY3F0dERSUlLd7by8PISHh2PgwIEIDg42yjXuPX9jiouLcfHiRaNcq5axntf+/fsxYsQIfPTRR/WOX7x4EcXFxUartynm+P201rFjx/DYY48BQINxaY8LFy5g06ZNiIuLg6enJ3Q6Hd5//3289957+POf/4yFCxca7VpEZDsYnolIVH5+fggKCkJmZiZ+/fVXKBQKVFZWwtXVFd9++y1iY2Pxww8/QK/Xw9PTE++++y569+6NvLw8vPXWW8jPz0fXrl1x69atunP2798fx48fR6dOnfCPf/wDCQkJsLOzQ1BQENavX481a9agqqoKERERiI+PR2ZmJj766CPcuXMHOp0OixYtQlRUFADgiy++wPbt2+Hp6YmgoCCjPa9NmzZh586dkMlk6NmzJ959910cP34cP/zwA3Q6HaqqqvDII49gz549eOmll7Bt2zbodDq4ublh5cqVddf58ccfcfDgQWzevBkA8Pvvv+OZZ55BamoqNm3ahH379kEul8PLywt/+tOf4Ovr2+bfT3BwcKN1+/j4YNGiRRgwYACUSiWKiooQERGBFStWQK1WY9asWTh37hwANLhd6/z58/j0009RXV2NgoICjB49Gh9//DE+//xz5Ofn480338Qnn3yCDRs24Mknn8T06dPx008/YePGjdDr9XBxccGaNWswaNAg/PWvf0VWVhYKCgqQlZUFPz8/fPrppw2ee0FBAQRBQFVVFQBAJpPh1VdfxdWrVwEAf/3rX1FUVIQlS5bghRdeqHtcYWEh7OzscOjQIeTl5eGDDz5ATk4OtFotZs6cWe++RGSDBCIiM1GpVMKQIUPqHTt79qwwfPhwITs7W4iLixOGDx8ulJaWCoIgCCdPnhSeeOIJoaKiQhAEQThy5Igwffp0QRAE4aWXXhI+//xzQRAEITMzUxgyZIgQFxcnCIIg9OvXT7h165bw008/CVOnThXu3LkjCIIgfPzxx8Lf/va3enVotVphxowZQnp6uiAIglBSUiKEhYUJ586dE/bt2yfMmDFDKC0tFbRarbB06VLhqaeeavfzUigUwmOPPSaUl5cLgiAIX375pbB48eK6f37//fcFQRCEuLg4YenSpQ2O36u0tFR46KGHhPz8fEEQBOGTTz4RPvvsMyE7O1sYNmyYoNFoBEEQhK+//lrYt29fu34/zdX91FNPCUuWLBGqq6uF4uJiYdq0acKBAwcanPPe2/c+v5UrVwonTpwQBEEQysrKhBEjRggXL14UBEEQJk6cKFy4cKHuOikpKcJvv/0mjB49Wrh586YgCIJw7Ngx4ZFHHhFKS0uFL7/8Upg8eXLdeC9btkz44osvGjzf6upq4fXXXxdCQkKEOXPmCO+//75w8OBBQa/XNznmN2/eFCZOnCicOnVKEARBWLRokbB//35BEAShqqpKWLRokbBz585mx5mIrBtnnonIrGpnfAFAp9PBy8sLn376Kbp06QLg7qyxq6srgLt9vjdu3MDjjz9e9/iSkhLcuXMHx44dwx/+8AcAQFBQEEaMGNHgWsePH8f06dPh4eEBAFizZg2Au7OftTIzM3Hz5k28/fbb9Wr89ddf8fvvv2PKlCl19URGRuLbb79t9/M6fPgw5s2bB2dnZwBAdHQ0Nm/ejOrqasMH8r9cXV0xZcoUJCcn45lnnsH27duxdetW+Pn5ITg4GHPnzsW4ceMwbtw4jBo1qsXzNfc8Wqr7scceg1wuh1wux/Tp03H06FH07dvXoOexfv16HD58GJs3b8a1a9eg0WhQUVHR5P1PnDiBkSNHIjAwEAAwatQodOrUqa43+uGHH64b7wEDBjTa8iKXy/HnP/8Zq1evxsmTJ3H69Gn84Q9/wKhRo/CXv/ylwf1v376NJUuW4PXXX8fw4cNRUVGB06dPo7i4uK4fvaKiApcuXcKMGTMMet5EZH0YnonIrO7vqb1fbTADAL1ej4iICKxatarudn5+Pjw8PCCRSCAIQt197ewa/jmTyWSQSCR1t0tKShp8kbC2FeLemgoLC+Hm5oZPPvmk3jVkMpnRnte9den1etTU1DT52JYsWLCgrp2ld+/edYHyu+++w8WLF3H8+HF8/PHHGDt2LFavXt3suZp7Hi3Vfe/vQBAESKXSBr8nrVbb6Lmfeuop9O/fH2PHjkVYWBjS0tLqPa6lWmqvWVuPo6Nj3fH7a6ilUCjg5eWFyZMnY/bs2Zg9ezZefPFFTJo0Cbdv365338rKSrzwwguYO3cuwsPD62oQBAHbtm2Dk5MTgLsB28HBocm6icj6cbUNIrJYY8aMwc6dO5Gfnw/g7uoHTz/9NABg7Nix+PHHHwEA2dnZOHnyZIPHjx49Gvv27UNZWRmAuz2s//73v2FnZwedTgdBENCzZ896gTEnJwfh4eFIT0/HuHHjsHv3bpSUlECv17f4RURDjR07FnFxcXUzq99++y2GDx8Oe3v7Jh8jk8maDNhDhgwBAGzatAnz588HAFy6dAnh4eHo3bs3li1bhmeeeabdX5Jsqe7k5GTo9XoUFxcjJSUFkyZNgru7O7RaLX777TcAwM6dOxuct6SkBBcvXsSbb76JqVOnIjc3Fzdv3oRer2/yuY8aNQpHjx6FSqUCcPdThpycHAwePNjg5yOVSrFhwwbk5ubWHbt69Sq6du1a92kFcPcN1muvvYbg4OB6q4O4urpiyJAh+Oabb+qex8KFC7F//36DayAi68OZZyKyWGPGjMGSJUuwePFiSCQSuLq6YuPGjZBIJFi3bh3WrFmDsLAw+Pv7N7oSxPjx4/Hbb7/VrZrQp08f/PGPf4STkxMGDRqEmTNnYuvWrfjb3/6Gjz76CP/6179QU1ODV199FaGhoQCAy5cvIzIyEu7u7ggODkZRUVG7n1dUVBRycnIwf/586PV6BAUFYcOGDc0+ZuTIkXjzzTfxxz/+Ee+++26Dn8+fPx9/+9vf8OijjwIAgoODERYWhsjISDg7O8PR0RFr164FgLoWg1dffdWodVdVVSEqKgrl5eV44okn6tpEVq1ahSVLlqBTp06YPn16g/O6u7tj6dKlmDt3LpydneHn54dhw4bhxo0bGDVqFKZMmYJVq1bhvffeq3tMnz59sG7dOixfvhw6nQ6Ojo7YvHkz3NzcDH4+8+bNQ2VlJZYsWYLq6mpIJBL06NEDX3/9db1PGVJSUpCamoqBAwdizpw5dbPYX331FTZs2IA//vGPmDVrFqqrqxEeHo7Zs2e3alyJyLpIhOY+FyMiIjLAokWL6lbBICKyZWzbICIiIiIyEGeeiYiIiIgMxJlnIiIiIiIDWc0XBvV6PcrLyyGXyxssT0REREREZCyCIECr1cLFxQVSaf25ZqsJz+Xl5bhy5YrYZRARERFRB9GvX78Gq/hYTXiWy+UA7j6J5tZCNbb09HQMHDjQbNfriDjGpscxNi2Or+lxjE2L42t6HGPTM+YYV1dX48qVK3X5815WE55rWzXs7e3NvnsTd4syPY6x6XGMTYvja3ocY9Pi+Joex9j0jD3GjbUK8wuDREREREQGMunM88aNG5GSkgLg7k5fq1evxpo1a6BUKuHk5AQAWL58OaZMmWLKMoiIiIiIjMJk4fnYsWM4evQoEhISIJFI8Pzzz2Pfvn1IT0/Hd999B19fX1NdmoiIiIjIJEwWnn18fPDWW2/Vfbmvd+/eyM7ORnZ2Nt5++23k5eVhypQpWL58eYMlQFpDr9ejsLAQd+7cgU6nM1b5dezs7JCRkWH089o6mUwGT09PeHt7t+v3S0RERGRJTBae+/btW/fPmZmZSElJwdatW3Hq1CmsW7cObm5uWLZsGRQKBRYsWNDm66jVakgkEvTo0cMka0CXl5fDxcXFqOe0dbVrI+bl5UGtVqN79+5il0RERERkFCbfnvvq1atYtmwZXnnlFcydO7fez/bt24fExERs2rSpxfNoNBqkp6c3OG5nZ4devXpxdtMC6fV6XLt2DTU1NWKXQkRERNRqAwcObLCCh0m/MKhUKrFixQq8/fbbmDlzJi5fvozMzExMmzYNwN0ZSju71pVw/5PIyMhosHi1MXHmuX3s7e0xePDgZu+jVCoRGhpqpoo6Jo6xaXF8TY9jbFocX9PjGJueMce4qUlbwIRL1eXk5ODll1/Ghg0bMHPmTAB3w/LHH3+M4uJiaLVa/Pjjj1xpg4iIiIishsnC89dffw2NRoP169cjIiICEREROHfuHJYuXYqFCxdi5syZCAkJQXh4uKlKEI1arcbAgQMRERGBOXPmYObMmXj22WeRm5vbpvPFx8fjrbfeAgAsWbIEeXl5Td73yy+/xJkzZwAA77zzDi5evNimaxIRERFRQyZr21i7di3Wrl3b6M+efPJJU13WYvj6+iIpKanu9vr16/HJJ5/gs88+a9d5//nPfzb789OnT2PEiBEAgI8++qhd1yIiIiKi+qxme25D5R9IRd7+A0Y7n06ng0wmAwD4TZ4E30kT2nSeESNG4LPPPsOkSZMwaNAgZGRk4Pvvv8eRI0fwn//8B3q9Hg888ADWrVsHBwcHJCYm4u9//ztcXV3RrVs3ODs7AwAmTZqELVu2wMfHB++//z6USiXkcjleeuklVFdXIz09HWvXrsXGjRvx4YcfYvny5RgxYgQ2b96M5ORkyGQyPPLII1i1ahVycnKwfPly9O3bFxkZGejcuTO++OILeHp6Gmn0iIiIiGwLl6gwA61Wiz179mDIkCEAgHHjxmHPnj24ffs2YmJisG3bNiQlJaFz5874+uuvkZeXhw0bNmDr1q348ccfUV5e3uCc3377LSoqKpCSkoJvvvkGmzZtwowZMzBw4EB8+OGH6N+/f919Dx06hAMHDiAuLg4JCQm4ceMGtm3bBgC4dOkSnn32WezYsQPu7u7Yvn27WcaEiIiIyBrZ3Myz76QJbZ4dbkxbV9vIz89HREQEAKC6uhqDBg3CG2+8gZ9//rlu9YmTJ0/ixo0bdetca7VaDBgwAOfOncPQoUPh7e0NAJg1axZOnDhR7/ynT5/GggULIJVK4ePjg507dzZZy4kTJzBz5sy6LdEjIyORmJiI8ePHo3PnzhgwYACAu2tzFxcXt/q5EhEREXUUNheeLcX9Pc/3ql1qT6fTISwsrK43vLy8HDqdDsePH8e9y283tpyfnZ1dvQ1hbty4gS5dujR6Pb1e3+BY7drL9y77J5FIYOJlv4mIiIisGts2RDRixAjs27cPt27dgiAIeO+99/Cf//wHoaGhOH/+PPLy8qDX67Fr164Gjx0+fDh27doFQRBw69YtPPXUU6iuroZMJmuwTfnIkSOxc+dOVFVVoaamBnFxcRg5cqS5niYRERFRq6QqVVj84V7MfiMJiz/ci1SlSuyS6nDmWUTBwcFYvnw5nn76aej1eoSEhGDp0qVwcHDA2rVr8cwzz8DJyQl9+vRp8NgnnngCH374IWbPng0AePfdd+Hq6oqxY8di3bp1+H//7//V3XfixInIyMhAZGQkampqMGbMGDz11FNtXjqPiIiIyFRSlSpsjE2DRnt3MrCgqBIbY9MAABNCA8UsDYAZtuc2ltqdXhrbYTAkJMRk1+UOg+1jyO+Huy6ZHsfYtDi+pscxNi2Or+lxjA23+MO9KCiqbHDcx8sJ/7d2apOPM8UOg41tz822DSIiIiKyGIWNBOfmjpsbwzMRERERWQxvL6dWHTc3hmciIiIishjRYSFwkMvqHXOQyxAdZro23dbgFwaJiIiIyGLUfilwS0oGCosq4e3lhOiwEIv4siDA8ExEREREFmZCaKDFhOX7sW2DiIiIiMhADM9ERERERAZi24aRvf/++zh79iy0Wi1u3ryJ3r17AwCio6MRGRnZrnMvWbIEH374Ifz8/Np1nv79+yM4OBgAUF1djd69e2PVqlUICgpq9nFr1qzB8uXL0a1bt3Zdn4iIiMhaddjwnKpUmaQRfd26dQAAtVqN6OhoJCUltfuctf75z38a7Vz31vXDDz/gueeew65du2Bvb9/kY06ePImXX37ZaDUQERERWZsOGZ7F2vZx0qRJGDRoEDIyMvD9999jy5YtOH78OIqLi+Hr64vPP/8c3t7eGDNmDKZNmwalUgmZTIa//OUvCAwMxKRJk7BlyxacOnUKR44cQXFxMVQqFR555BG89957AIA///nP2LNnD7y8vODj44NJkyZh3rx5zda1cOFCfPfddzhy5AgmT56Mzz//vEFd8fHxyM/Px9KlS7F161acOHEC33zzDaqqqlBdXY2PP/4Yw4YNM9nYEREREVmCDtnzvCUloy4419JoddiSkmHya48bNw579uxBWVkZrl27hm3btmHPnj3o0qULkpOTAQAFBQUYNWoUEhMTMXz4cGzdurXBec6dO4cvv/wSycnJOHjwIC5fvowDBw5AqVRix44d+Oqrr/Drr78aXFefPn1w7do13Lhxo9G6li5dCl9fX3z11Vfw8PDAtm3bsHnzZiQnJ+P555/HV199ZbQxIiIiIrJUHXLmWcxtHwcPHgwACAoKwh/+8AfExsbi+vXrOH/+PLp37153v7FjxwIA+vbtizNnzjQ4z9ChQ+Hq6goACAwMRHFxMY4dO4awsDDY29vD3t4ejz76qMF1SSQSODo6tlgXAEilUmzatAkHDhzA9evXcerUKUilHfJ9GBEREXUwHTLxiLnto4ODAwAgPT0dzz33HPR6PaZNm4ZHH30UgiA0uJ9EIql3/P6f33sfqVQKvV7fprouX76MPn36tFgXAJSXlyMqKgpqtRrDhw/HokWL2nRNIiIiImvTIcOzJWz7ePr0aTz88MNYuHAhevTogdTUVOh0upYf2IzRo0dj7969qK6uRllZGVJTUyGRSFp83Pfffw+JRIIRI0Y0W5dMJoNOp0NmZiYkEgleeOEFjBgxAvv27Wt37URERETWoEO2bVjCto8zZszA8uXLMWvWLADAwIEDoVar23XOCRMm4Ny5c5g7dy48PDzg6+tbb4b6XhEREQAAvV6PwMBA/POf/4RUKm22rgkTJmDp0qX45z//iZCQEISFhUEikWDMmDFQKpXtqp2IiIjIGkiExnoCLJBGo0F6ejoGDhxYLxBmZGQgJMR0M8bl5eVwcXEx2fmN6dy5c8jMzMTcuXOh1Wrx2GOP4eOPP65b01kMhvx+lEolQkNDzVRRx8QxNi2Or+lxjE2L42t6HGPTM+YYN5U7gQ4682yrevbsiY0bN+Kbb76BIAiYM2eOqMGZiIiIyNYwPNsQT09PfP3112KXQURERGSzbOILg21dYYJMi78XIiIisjVWP/Ps4uKCrKws+Pn5QS6XG7S6BJmWIAjQarXIy8uzmn5xIiKyfqlKlaiLAVDHYPXhOSAgAIWFhbhx4wZqamqMfv7q6mrY29sb/by2zs7ODh4eHvD29ha7FCIi6gBSlSpsjE2r20G4oKgSG2PTAIABmozK6sOzVCqFr68vfH19TXJ+pVJZtysgERERWaYtKRl1wbmWRqvDlpQMhmcyKpvoeSYiIqKOrbCoslXHidqK4ZmIiIisnreXU6uOE7UVwzMRERFZveiwEDjIZfWOOchliA4z3UZq1DFZfc8zERERUW1fM1fbIFNjeCYiIiKbMCE0kGGZTI7hmYiIiIgsSunV35AVlwB7787o9fxiscuph+GZiIiIiCxCScYlqGIUuHP2HGQuLuj53DNil9QAwzMRERERiUYQBBRfuAhVjAIl6b9A7uGOoOin4B82DXbOzmKX1wDDMxERERGZnSAIKFKehTpGgdLLV2DfqRN6Pvcs/KZNgczBQezymsTwTERERERmI+j1uH3yFFQxCpRfuw4HXx/0emEp/CZPhNTeXuzyWsTwTERERCSyVKXK5pfZE3Q6FB49BlWsApUqNRy7dkGfFS/DZ/w4SO2sJ5JaT6VERERENihVqcLG2DRotDoAQEFRJTbGpgGATQRovVaLgkOHoVbEoyonF87dA9Hvjdfg/choSGSylk9gYRieiYiIiES0JSWjLjjX0mh12JKSYdXhWV9djbyfDiArPgGagkK49O6F4LdWodOIhyGRWu8m1wzPRERERCIqLKps1XFLp6uqQu6efchKSIK2qAhu/fuj94vL4DlsKCQSidjltRvDMxEREZGIvL2cUNBIUPb2chKhmrarqahA7q7dyE7eDm1xCTweHIiA11+Fx4MDbSI012J4JiIiIhJRdFhIvZ5nAHCQyxAdFiJiVYbTlpYiZ/tOZO/YBV15OTyHDUXggii4hwSLXZpJMDwTERERiai2r9naVtuovlOM7KRk5OzaDX1VFTqNeBiBC6Lg2qe32KWZFMMzERERkcgmhAZafFiupbl1C1kJScjbsw96rRbeY0YjICoSLj2CxC7NLBieiYiIiKhFVXn5yIpPQN5PByDo9fAZPw4BUfPgHNBN7NLMiuGZiIiIiJpUmZUNtSIe+amHIJFK4Tt5EgIi58DRz0/s0kTB8ExEREREDZTfuAm1Ig6FR49BameHLjPD0G1uBBw6dxa7NFExPBMRERFRnbLffocqNg63T5yE1NER3ebMRteIWbD39BS7NIvA8ExERERkIVKVKtFW3SjJuAR1rAJFynOQubgg8PEF6BI+A3I3N7Nc31owPBMRERFZgFSlqt56zwVFldgYmwYAJgvQgiCg+GI61DEKFF9Mh527O4IWPQn/GdNh5+xskmtaO4ZnIiIiIguwJSWj3kYpAKDR6rAlJcPo4VkQBBQpz0IdE4fSy5ch9/JCj8VPw3/aVMgcHY16LVvD8ExERERkAQob2aK7ueNtIej1uH3yFFQxCpRfuw4HH2/0emEJ/CZPgtTe3mjXsWUMz0REREQWwNvLCQWNBGVvL6d2n1vQ6VD48zGoY+NQcVMFxy7+6PPKS/AZPw5Subzd5+9IGJ6JiIiILEB0WEi9nmcAcJDLEB0W0uZz6mtqUJB6GOq4eFRl58ApMAB9V74Kn7GPQCKTGaPsDofhmYiIiMgC1PY1G2O1Db1Wi7yfDiArPgGa/AK49OqJ4LdWodOIhyGRSo1deofC8ExERERkISaEBrbry4E6jQZ5e/chKz4J1bdvw61/P/RatgReocMgkUiMWGnHxfBMREREZOVqKiqQm7IH2UnJ0BaXwH3gA+j72ivwGPQgQ7ORMTwTERERWamasjJk79iFnO07UVNWBs9hQxE4PxLuA9reJ03NY3gmIiIisjLVd4qRnbwdubt2Q1dZiU4jhiNgfhTc+vYRuzSbx/BMREREZCU0t24jKyEJeXv2Qq/VwvuR0QiYPw8uPXqIXVqHwfBMREREZOGq8vORFZ+IvH37Iej18J0wDt0i58E5oJvYpXU4DM9EREREFqoyOxtqRQIKUg8BEgl8J09CQOQcOPr5iV1ah2XS8Lxx40akpKQAAMaPH4/Vq1fj2LFj+NOf/gSNRoOwsDCsXLnSlCUQERERWZ3yGzehVsSh8OgxSO3s4D99GrrNmwMH785il9bhmSw8Hzt2DEePHkVCQgIkEgmef/557NixAxs2bMC3336LLl26YNmyZTh06BDGjx9vqjKIiIiIrEbZ79egilHg9omTkDo6otuc2egaMQv2np5il0b/ZbLw7OPjg7feegv29vYAgN69eyMzMxNBQUEIDLy7+PesWbOwe/duhmciIiLq0EouXYY6RoEi5VnIXJwRsCAKXWeFQ+7uJnZpdB+JIAiCqS+SmZmJhQsX4qmnnsL169exYcMGAHdnp//1r3/h//7v/1o8h0ajQXp6uqlLJSIiIjILQRCgz7wB3ZGfoc+8ATg5wW7kw5AND4XE0VHs8gjAwIED4eDgUO+Yyb8wePXqVSxbtgyrV6+GTCZDZmZm3c8EQWj1rjeNPQlTUiqVCA0NNdv1OiKOselxjE2L42t6HGPT4via3r1jLAgC7pw7D1WsAqUZlyD38kT3xU/Df+oUyJycRK7UehnzddzcpK1Jw7NSqcSKFSvw9ttvY+bMmTh16hQKCgrqfl5QUABfX19TlkBERERkEQS9HrdPnYYqJg7lv/8Oe29v9Fr6PHwfnQSZGScGqX1MFp5zcnLw8ssv4/PPP8eoUaMAAIMHD8b169dx48YNBAQEYMeOHYiMjDRVCURERESiE3Q66H75Fef/8x0qbtyEo78/er/8InwnjodULhe7PGolk4Xnr7/+GhqNBuvXr6879vjjj2P9+vV45ZVXoNFoMH78eEyfPt1UJRARERGJRl9Tg8LDR6CKjYc2Oxt2AQHou/JV+Ix9BBKZTOzyqI1MFp7Xrl2LtWvXNvqz5ORkU12WiIiISFR6rRb5+w9CHZcATX4+XHr2hHz+PAx9YiEkUqnY5VE7cYdBIiIiIiPQaTTI27sPWQlJqL51G679+qLX0ufg9VAozp49y+BsIxieiYiIiNqhpqISubv3IDsxGdriYrgPfAB9VyyHx+BBrV5VjCwfwzMRERFRG9SUlSF7xy7kbN+JmrIyeA4dgoD5kfB4YIDYpZEJMTwTERERtYK2uBjZyTuQs2s3dBUV8Br+EAIXRMGtX1+xSyMzYHgmIiIiMkD17SJkJSYhd/de6Kur0Xn0KATOj4RLzx5il0ZmxPBMRERE1Iyq/HxkxSch76f9EHQ6+Iwfh4CouXAOCBC7NBIBwzMRERFRIypzcqBWxKPg4CFAIoHvxAnoFjkXTl38xS6NRMTwTERERHSPipuqu6H5yFFI7ezgP30aus2NgIOPt9ilkQVgeCYiIiICUHbtGtQxcbh1/ASkjo7oOjsc3ebMhr2Xl9ilkQVheCYiIqIOrfTyFahiFCg6o4TM2RkBC6LQddZMyN3dxS6NLBDDMxEREXU4giCg5JdfoYpRoDjtAuzc3ND9yYXoMiMMdq4uYpdHFozhmYiIiDoMQRBw59x5qGPjUPJrBuSenujx7NPwnzYFMicnscsjK8DwTERERDZP0Otx+/QZqGMUKPvtd9h7e6PX0ufg++hkyBwcxC6PrAjDMxEREdksQafDreMnoIqNQ0XmDTj6+6HP8hfhM2E8pHK52OWRFWJ4JiIiIpujr6lB4eGjUCviUJmVDaeAbui7cgV8xo6BRCYTuzyyYgzPREREZDP0Wi3yDxyEOi4Bmrx8uPTsgf6r30DnUSMhkUrFLo9sAMMzERERWT2dRoO8vT8hKyER1bduw7VvX/Ra8hy8HgqFRCIRuzyyIQzPREREZLVqKiqRu3sPshOToS0uhvsDA9B3xXJ4DB7E0EwmwfBMREREVqemrBw5O3che/sO1JSWwXPIYAQsiITHAw+IXRrZOIZnIiIishrakhJkJ+9Azs4U6Coq4DX8IQQuiIJbv75il0YdBMMzERERWbzq20XISkpGbsoe6Kur0Xn0SARERcK1V0+xS6MOhuGZiIiILJamoADq+ETk7dsPQaeDz7ixCIiaB+fAALFLow6K4ZmIiIgsTmVOLtSKeBQcTAUkEvhOnIBukXPh1MVf7NJsXqpShS0pGSgsqoS3lxOiw0IwITRQ7LIsBsMzERERWYwKlRpqRRwKDh+FRCaD//Sp6DY3Ag4+PmKX1iGkKlXYGJsGjVYHACgoqsTG2DQAYID+L4ZnIiIiEl3ZtetQx8bh1vETkNrbo+vscHSLmA37Tl5il9ahbEnJqAvOtTRaHbakZDA8/xfDMxEREYmm9MpVqGIUKDp9BjJnZwREzUPX2eGQu7uLXVqHVFhU2arjHRHDMxEREZld8S+/QB0Thzvn02Dn5oruTy5ElxlhsHN1Ebu0Ds3bywkFjQRlby8nEaqxTAzPVorN/EREZG0EQcCd82lQxyhQ8msG5J6e6PFMNPymTYWdM8OZJYgOC6nX8wwADnIZosNCRKzKsjA8WyE28xMRkTURBAG3T52BOlaBsqu/wb5zZ/Ra+hx8H50MmYOD2OXRPWpzBCfomsbwbIXYzE9ERNZA0Olw68RJqGIUqMi8AUd/P/R++UX4ThwPqVwudnnUhAmhgcwTzWB4tkJs5iciIksm6HQoOHwEakU8KtVZcOrWFX1fewU+48ZCIpOJXR5RuzA8WyE28xMRkSXSa7XIP5iKrLgEVOXmwblHEPqvfgOdR45gaCabwfBshdjMT0RElkSn0SBv335kxSei+tYtuPbpjeC3n0Wn4aGQSKVil0dkVAzPVojN/EREZAl0lZXI3b0XWYnJ0N65A/cBIeiz/EV4Dh0CiUQidnlEJsHwbKXYzE9ERGKpKStHzq4UZCdvR01pGTwGD0LgqtfhMfABsUsjMjmGZyIiIjKItqQE2dt3ImfHLugqKuA1PBSB86Pg1r+f2KUB4B4IZB4Mz0RERNSs6qIiZCUmI3f3Xug1GnQeNRIB8+fBtVcvsUurwz0QyFwYnomIiKhRmoJCZCUkInfvTxB0OviMHYOAqHlw7m55YZR7IJC5MDwTERFRPVW5uVArEpB/MBUQBPhMnICAqLlw6tJF7NKaxD0QyFwYnomIiAgAUKFSQ62IR8HhI5DIZPCb8ii6zYuAo6+v2KW1iHsgkLkwPBMREXVw+rx8XPpkA24dOwGpvT26zpqJrhGz4dC5k9ilGYx7IJC5MDwTERF1UKVXrkIdG4fqU6dxx8kJAZFz0XV2OOQeHmKX1mrcA4HMheGZiIiogyn+5VeoYxS4cz4Ndq6usBs/FqFLn4edq6vYpbUL90Agc2B4JiIi6gAEQUBx2gWoYhQo+eVXyD08EPT0IvhPn4a0jF+tPjgTmQvDMxERkQ0TBAFFp89AFROHsqtXYd+5E3o+vxh+Ux+FzMFB7PKIrA7DMxERkQ0S9HrcOn4S6lgFyq9nwsHXF71fXAbfyRMhlcvFLo/IajE8ExER2RBBp0PBkaNQx8ajUq2GY9eu6PvqcniPGwupHf+zT9Re/LeIiIjIBui1WuQfPISsuHhU5ebBOag7+r35OrxHj4REJhO7PCKbwfBMRERkxXQaDfJ/OgB1fCKqCwvh2qc3gt9+Bp2GPwSJVCp2eUQ2h+GZiIjICumqqpC7ey+yEpOgLboDt5Bg9Hn5BXgOHQKJRCJ2eUQ2i+GZiIjIitSUlyNn125kJ21HTWkpPAY9iMA3X4f7AwMYmonMgOGZiIjICmhLSpG9fQdydu6CrrwCXg+FImB+JNyD+4tdGlGHwvBMRERkwaqLipCdtB05KXugr6pC51EjEbAgEq69eoldGlGHxPBMRERkgTSFt5AVn4i8fT9BX1MD7zGPIHD+PDh37y52aUQdGsMzERGRBanKzYU6PhH5+w8CggCfieMREDkXTl27il0aEYHhmYiIyCJUqLOQFReP/NTDkEil8JvyKLrNi4Cjr6/YpRHRPRieiYiIRFSemQl1bDwKfz4GqVyOruEz0HVOBBw6dxK7NCJqBMMzERG1KFWpwpaUDBQWVcLbywnRYSGYEBoodllWrfTqb1DHKnD75GnInJwQEDkXXWeHQ+7hIXZpRNQMhmciImpWqlKFjbFp0Gh1AICCokpsjE0DAAboNij5NQOqGAXunDsPO1dXBC58DF3DZ8DO1VXs0ojIAAzPRETUrC0pGXXBuZZGq8OWlAyGZwMJgoDitAtQxcahJP0XyD3cERT9FPzDpsHO2Vns8oioFRieiYioWYVFla06Tv8jCAKKziihilGg7MpV2HfqhJ7PPwu/qVMgc3AQuzwiagOGZyIiapa3lxMKGgnK3l5OIlRjHQS9HrdOnIQ6Jg7l16/DwdcXvV9cBt/JEyGVy8Uuj4jageGZiIiaFR0WUq/nGQAc5DJEh4WIWJVlEnQ6FBz5GerYOFSq1XDs2gV9VrwMn/HjILXjf3KJbAH/TSYiombV9jVztY2m6bVaFKQeglqRgKrcXDgHdUe/N1bC+5FRkMhkYpdHREbE8ExERC2aEBrIsNwIfXU18n46gKz4BGgKCuHSuzeC16xGp4eHQyKVil0eEZmAScNzWVkZHn/8cWzevBkBAQFYs2YNlEolnJzu9sktX74cU6ZMMWUJRERERqerqkLunr3ISkiCtugO3IL7o/dLL8Bz6BBIJBKxyyMiEzJZeE5LS8PatWuRmZlZdyw9PR3fffcdfLnVKBERWaGa8nLk7NqN7OQdqCkpgcegBxHw+mvweHAgQzNRB2Gy8BwTE4N169Zh9erVAIDKykpkZ2fj7bffRl5eHqZMmYLly5dDyo+1iIjIwmlLSpGzYyeyd+yCrrwcXqHDEDA/Eu4hwWKXRkRmJhEEQTDlBSZNmoQtW7ZAEASsX78e69atg5ubG5YtW4bw8HAsWLDAoPNoNBqkp6ebslQiIqJ6hLIy1Jw4Bd2Zs0B1NaT9+8Fu7COQdu0idmlEZAYDBw6Ew31rspvtC4OBgYHYtGlT3e1FixYhMTHR4PBcq7EnYUpKpRKhoaFmu15HxDE2PY6xaZlqfFOVKq5w8V/mfg1rCm8hKyEReXt/gr6mBt5jRiMgKhIuQd3NVoM58W+E6XGMTc+YY9zcpK3ZwvPly5eRmZmJadOmAbi765Id17wkImpUqlJVb23lgqJKbIxNA4AOG6DNoSovD+q4ROTvPwAIAnwmjEdA5Fw4desqdmlEZCHMll4FQcDHH3+MkSNHwtnZGT/++CPmzp1rrssTEVmVLSkZ9TYlAQCNVoctKRkMzyZQmZUNtSIO+amHIZFK4ffoJHSbNxeOfvyCOxHVZ7bwHBwcjKVLl2LhwoWoqanB1KlTER4ebq7LExFZlcJGtsNu7ji1TXnmDahj41D48zFI5XJ0mTkD3ebOhkPnzmKXRkQWyuTh+cCBA3X//OSTT+LJJ5809SWJiKyet5cTChoJyt5eTiJUY3tKr/4GdawCt0+ehtTREd3mRqBrxGzYe3qIXRoRWTg2HRMRWaDosJB6Pc8A4CCXITosRMSqrF9JxiWoYhS4c/YcZC4uCHx8AbqEz4DczU3s0ojISjA8ExFZoNq+Zq620X6CIKD4wkWoYhQoSf8Fdu7uCFr0JPxnTIeds7PY5RGRlWF4JiKyUBNCAxmW20EQBBQpz0Ido0Dp5SuQe3mhx+Jn4D9tCmSOjmKXR0RWiuGZiIhsiqDX4/bJU1DFKFB+7TocfLzR64Ul8Js8CVJ7e7HLIyIrx/BMREQ2QdDpUHj0GFSxClSq1HDs4o8+r7wMnwnjIOW+AkRkJPxrQkREVk1fU4OC1MNQx8WjKjsHzt0D0e+N1+D9yGhIZDKxyyMiG8PwTEREVklfXY28/QeQFZ8ITX4BXHr1RPBbq9FpxHBIpFKxyyMiG8XwTEREVkVXVYXcPfuQlZAEbVER3Pr3Q69lS+AVOgwSiUTs8ojIxjE8ExGRVaipqEDurt3ITt4ObXEJPB4ciIDXX4XHgwMZmonIbBieiYjIomlLS6FNPYwzf/4CuvJyeA4bisAFUXAPCRa7NCLqgBieiYjIIlXfKUZ2UjJydu2GvqoKnUY8jMAFUXDt01vs0oioA2N4JvqvVKWKu7kRWQDNrVvISkhG3p690Gu18B4zGiUDQhAyI0zs0oiIGJ6JgLvBeWNsGjRaHQCgoKgSG2PTAIABmshMqvLykRWfgLyfDkDQ6+E7YRy6Rc6Dc0A3KJVKscsjIgLA8EwEANiSklEXnGtptDpsSclgeCaj4acbjavMyoZaEY+CQ4cBiQS+kychIHIOHP38xC6NiKgBhmciAIVFla06TtRa/HSjofIbN6FWxKHw6DFI7ezgP2M6us2JgIN3Z7FLIyJqEsMzEQBvLycUNBKUvb2cRKiGbBE/3fifst9+hyo2DrdPnITU0RHd5sxG14hZsPf0FLs0IqIWMTwTAYgOC6k3KwgADnIZosNCRKyKbAk/3QBKLl2GOkaBIuVZyFxcEPjYfHQJnwm5u5vYpRERGYzhmQj/+9ic/ahkKh310w1BEFCS/gtUMQoUX7gIO3d3BC16Ev5h02Dn4iJ2eURErcbwTPRfE0IDGZbJZDrapxuCIODOufNQxShQmnEJci9P9Fj8DPynTYHM0VHs8oiI2ozhmYjIDDrKpxuCXo/bp05DFROH8t9/h4OPN3q9sAR+kydBam8vdnlERO3G8ExEZCa2/OmGoNOh8OfjUCviUHHjJhy7+KPPKy/BZ/w4SOVyscsjIjIahmciImozfU0NCg4dhloRj6rsHDgFBqDf66/Be8xoSGQyscsjIjI6hmciImo1vVaLvJ8OICs+AZr8Arj06ongt1ah04iHIZFKxS6PiMhkGJ6JiMhgOo0GeXv3ISs+CdW3b8O1X1/0WrYEXqHDIJFIxC6PiMjkGJ6JiKhFNRWVyE3ZjeykZGiLS+A+8AH0fe0VeAx6kKGZiDoUhmciImpSTVkZsnfsQs72nagpK4Pn0CEImB8JjwcGiF0aEZEoGJ6JiKgBbXExspN3IGdnCnSVleg0YjgC5kfBrW8fsUsjIhIVwzMREdXR3LqN7MQk5O7eC71Wi86jRyFwQSRcevQQuzQiIovA8ExERKjKz0dWfCLy9u2HoNfDZ/w4BETNhXNAgNilERFZFIZnIqIOrDI7G2pFAgpSDwESCXwnT0TAvDlw9PcXuzQiIovE8ExE1AFV3LwJVWw8Co/+DKmdHfynT0W3uXPg4OMtdmlERBaN4ZmIqAMp+/0a1LEK3Dp+ElJHR3SLmIWuEbNg7+UldmlERFaB4ZmIqAMouXQZ6tg4FJ1RQubijIAFUeg6KxxydzexSyMisioMz0RmlqpUYUtKBgqLKuHt5YTosBBMCA0UuyyyQYIgoCT9F6hiFCi+cBF2bm7o/uRCdJkZBjsXF7HLIyKySgzPRGaUqlRhY2waNFodAKCgqBIbY9MAgAGajEYQBNw5dx6qGAVKMy5B7uWJHoufhv/UKZA5OYldXosae4PJ+XEishQMz0T3MeXM8JaUjLrgXEuj1WFLSgbDM7WboNfj9ukzUMcoUPbb77D39kavpc/D99FJkDk4iF2eQZp6gznzIXeEhopcHBERGJ6J6jH1zHBhUWWrjhMZQtDpUHjsBNSxClTcuAlHf3/0fvlF+E4cD6lcLnZ5rdLUG8z9aSV4NkqkooiI7sHwTK1mCz27TT0HU88Me3s5oaCRoOztZfkfpZPl0dfUoPDwEahi41GVnQ2ngAD0XfkqfMY+AolMJnZ5bdLUG8niCl2jx4mIzI3hmVrFFnp2m3sOpp4Zjg4LqXdtAHCQyxAdFmKU81PHoNdqkb//INRxCdDk58OlZw/0X/0mOo8aAYlUKnZ57dLUG0wPZ+t8M0BEtofhmVrFFnp2m3sOpp4Zrh0ja5+5J3HoNBrk7f0JWQmJqL51G659+6LXksXwGv4QJBKJ2OUZRVNvMCcPdhexKiKi/2F4plaxhZ7d5p7D608MM/nM8ITQQIZlapWaikrk7t6D7MRkaIuL4f7AAPRdsRwegwfZTGiu1dQbTDfki1wZEdFdDM/UKrbQs9vcc+DMMFmSmrJy5OzcheztO1BTWgbPIYMRsCASHg88IHZpJtXYG0ylkuGZiCwDwzO1ii307Lb0HDgzTGLTFhcjO3kHcnbthq6iAl7DH0Lggii49esrdmlERB0ewzO1ii3MzNrCcyDbVH27CFmJScjdvRf66mp0Hj0KgfMj4dKzh9ilERHRfzE8U6vZwsysLTwHsh2aggKo4xKR99N+CDodfMaNRUDUPDgHBohdGhER3YfhmYhIJJU5OVArElBwMBWQSOA7cQK6Rc6FUxd/sUsjIqImGLQg6F/+8pcGxz788ENj10JE1CFU3FThymdf4OxLK1B4+Aj8p09D6OZN6LP8RQZnIiIL1+zM85dffomSkhLs2rULZWVldce1Wi2OHj2KtWvXmrxAIiJbUXbtOtSxcbh1/ASkDg7oFjELXSNmwd7LS+zSiIjIQM2G58GDB+PixYuQSqXw9PSsOy6TybBhwwZT10ZEZBNKL1+BKlaBotNKyJydERA1D11nh0Puzo0/iIisTbPhefz48Rg/fjzGjRuHQYMGmasmIiKbUJz+C1QxChSnXYCdmxu6P7kQXWaEwc7VRezSiIiojZoNzx999BHeeecd/O1vf2v055s3bzZJUURE1koQBNw5dx7q2DiU/JoBuacnejwTDf/pUyFzsp7NhIiIqHHNhuegoCAAwLRp08xSDBGRtRL0etw+fQbqGAXKfvsd9p07o+eS5+A3ZTJkDg5il0dEREbSbHhOSEjAU089hWPHjuHTTz81V01ERFZD0Omg++VXnN+yFRWZN+Do74feL78I34njIZXLxS6PiIiMrNnwXFJSgnfffRc///xzo0vTcbUNIuqoBJ0OBYeOQK2IgzYrG3YB3dD3tVfgM24sJDKZ2OUREZGJNBue//rXv2L//v0NVtsgIuqo9Fot8g+mQq2IhyYvH849giCPnIOhTz7B0ExE1AE0G56Dg4MRHByM7t27Y9asWaipqYEgCJDzo0gi6mB0Gg3y9u1HVnwiqm/dgmvfPuj1/GJ4DX8IZ8+eZXAmIuogDNqee9SoUXj++edx4sQJ6HQ6DB8+HJ9++in8/PxMXR8Rkah0lZXI3b0XWYnJ0N65A/cBIei74mV4DB4EiUQidnlERGRmBoXnP/7xjxgyZAg+++wz6HQ6fPvtt3jvvffw97//3dT1ERGJoqasHDm7UpCdvB01pWXwGDwIgatfh8cDD4hdGhERicig8JyZmYkvvvii7vaKFSswc+ZMkxVFRCQWbUkJspN3IGdnCnQVFfAa/hAC50fCrX8/sUsjIiILYFB4rqmpgUajgcN/1yqtrKzkx5VEZFOqbxchKykZubv3Qq/RoPOokQiYHwnXXj3FLo2IiCyIQeF5xowZeOaZZzBv3jxIJBLExcVx4xQisgmaggJkJSQhd+9PEHQ6+Iwdg4CoeXDuHih2aUREZIEMCs8vv/wy/P39ceTIEej1esybNw9RUVGmro2IyGQqc3KRFZeA/IOpAADfiRPQLXIOnLp0EbUuIiKybAaF56effhr/+c9/EBkZaep6iIhMqkKlhloRh4LDRyGRyeA/bQq6zY2Ag4+P2KUREZEVMCg8l5aWoqKiAs7Ozqauh4jIJMquXYc6Ng63jp+A1N4eXWfNRLc5EbDv5GW2GlKVKmxJyUBhUSW8vZwQHRaCCaFsDyEisiYGhWcnJydMnDgR/fv3rxegN2/e3OzjysrK8Pjjj2Pz5s0ICAjAsWPH8Kc//QkajQZhYWFYuXJl+6onImpB6eUrUMXGoej0GcicnREQNQ9dZ4dD7u5u1jpSlSpsjE2DRqsDABQUVWJjbBoAMEATEVmRFsPzlStXMHnyZIwZMwb+/v4GnzgtLQ1r165FZmYmAKCqqgpvv/02vv32W3Tp0gXLli3DoUOHMH78+DYXT0TUlOJffoE6Jg53zqfBzs0V3Z94HF1mzoCdq4so9WxJyagLzrU0Wh22pGQwPBMRWZFmw3NcXBz+3//7fwgKCsLNmzexYcMGjB071qATx8TEYN26dVi9ejUA4MKFCwgKCkJg4N3/SMyaNQu7d+9meCYioxEEAXfOp0Edo0DJrxmQe3gg6OlF8J8+DXbOTqLWVlhU2arjRERkmZoNz99++y22b98OPz8/nDt3Dp9//rnB4fmjjz6qdzs/Px8+93whx9fXF3l5eW0omYioPkEQcPvUGahjFSi7+hvsO3dCz+cXw2/qo5D9d316sXl7OaGgkaDs7SVuqCciotZpsW3Dz88PADB06FAUFRW1+UJ6vb7exiqCILRpo5X09PQ219BWSqXS7NfsaDjGpmeLYyzo9dBfuoyaIz9DyMuHxNMTdjPDIBn8IHLs7JBjxr8XLY3v2GBHbD9VBa1OqDsml0kwNtjRJn83psBxMi2Or+lxjE3PHGPcbHi+P9zKZLI2X8jf3x8FBQV1twsKCuDr69vq8wwcOLBup0NzUCqVCA0NNdv1OiKOsenZ2hgLOh0KDh+BWhEPjToLTt26IuDV5fAeNxZSO4O+B21UhoxvaCjQsydX22grW3sNWxqOr+lxjE3PmGOs0WianLBt1X9l2rMl9+DBg3H9+nXcuHEDAQEB2LFjB9eNJqJW0Wu1yD94CFlx8ajKzYNzUHf0X/U6Oo8aCUk73tyby4TQQIZlIiIr12x4vnz5MoYNG1Z3u6qqCsOGDatruTh79qzBF3JwcMD69evxyiuvQKPRYPz48Zg+fXrbKyeiDkOn0SD/p/1QxyehurAQrn16I/jtZ9FpeCgkUqnY5RERUQfSbHjet29fuy9w4MCBun8eNWoUkpOT231OIuoYdJWVyN29F1mJydDeuQP3ASHo8/IL8Bw6pF2fhBEREbVVs+G5W7du5qqDiKhOTVk5cnalIDt5O2pKy+AxeBACV70Oj4EPiF0aERF1cOb/Zg0RURO0JSXI3r4TOTt3QVdeAa+HQhEwPxLuwf3FLo2IiAgAwzMRWYDqoiJkJSYjd/de6Kuq0HnUSATMj4Rr715il0ZERFQPwzMRiUZTUIishCTk7fsJ+poa+Iwdg4CouXDu3l3s0oiIiBrF8GwGqUqu7Up0r6rcXKjjEpB/IBUQBPhMHI+AyLlw6tpV7NKIiIiaxfBsYqlKFTbGpkGj1QEACooqsTE2DQAYoKnDqVCroY6NR8HhI5DIZPCbMhnd5s2BYxs2TCIiIhIDw7OJbUnJqAvOtTRaHbakZDA8U4dRnpkJVUwcbh07Dqlcjq7hM9B1TgQcOncSuzSrxk+1iIjMj+HZxAqLKlt1nMiWlF65CnVsHG6fOg2ZkxMCIuei6+xwyD08xC7N6vFTLSIicTA8m5i3lxMKGgnK3l5OIlRDZB7Fv/wKdYwCd86nwc7VFYELH0PX8Bmwc3UVuzSbwU+1iIjEwfBsYtFhIfVmhwDAQS5DdFiIiFURGZ8gCChOuwBVjAIlv/wKuYcHgp5eBP/p02DnzDeLxsZPtYiIxMHwbGK1M0DsSyRbJQgCis4ooYpRoOzKVdh37oSezy+G39RHIXNwELs8m8VPtYiIxMHwbAYTQgMZlsnmCHo9bh0/CXWsAuXXM+Hg64veLy6D7+SJkMrlYpdn8/ipFhGROBieiahVBJ0OBUd+hjo2DpVqNRy7dkXfV5fDe9xYSO34J8Vc+KkWEZE4+F86IjKIXqtFQeohqBUJqMrNhXNQd/R783V4jx4JiUwmdnkdEj/VIiIyP4ZnImqWvroaefv2Qx2fiOrCQrj26Y3gNavR6eHhkEilYpdHRERkVgzPRNQoXVUVcvfsRVZCErRFd+AW3B99Xn4BnkOHQCKRiF0eERGRKBieiaiemvJy5Ozajeyk7agpLYXHoAcR+MZKuA98gKGZiIg6PIZnIgIAaEtKkb19B3J27oKuvAJeocMQMD8S7iHBYpdGRERkMRieiTq46jt3kJ20HTm7dkNfVYXOo0YgYH4UXHv3Ers0IiIii8PwTNRBaQpvISshEXl7f4K+pgbeYx5B4Px5cO7eXezSiIiILBbDM1EHU5WbC3V8IvL3HwQEAT4TxiMgai6cunYVuzQiIiKLx/BM1EFUqLOgVsSj4NBhSKRS+E2ZjG5z58DRz1fs0owmVanipiFERGRSDM9ENq488waq4xJw7tdLkMrl6Bo+A13nRMChcyexSzOqVKWq3nbVBUWV2BibBgAM0EREZDQMz0Q2qvTqb1DHKnD75GnA3h7d5s1B19mzYO/pIXZpJrElJaMuONfSaHXYkpLB8ExEREbD8ExkY0oyLkEVo8Cds+cgc3FB4MLHkB/QFT3GjGlwX1tqcygsqmzVcSIiorZgeCayAYIgoPjCRahiFChJ/wVyD3cERT8F/7BpsHN2RoFS2eAxttbm4O3lhIJGgrK3l5MI1RARka1ieCayYoIgoEh5FuoYBUovX4F9p07o+dyz8Js2BTIHh2Yfa2ttDtFhIfXeDACAg1yG6LAQEasiIiJbw/BMZIUEvR63T56CKkaB8mvX4eDri94vLoPv5ImQyuUGncPW2hxqA7+ttKEQEZFlYngmsiKCTofCo8egilWgUqVGsYMHjvqORl7gACzyCYG/gcEZsM02hwmhgQzLRERkUgzPRFZAr9Wi4NBhqBXxqMrJheDjj5Su43HRKRCCRAoUV7e6X5ltDkRERK3H8ExkwfTV1cj76QCy4hOgKSiES+9eCH5rNVb9dAcFd6rq3be1/cpscyAiImo9hmciC6SrqkLunr3ISkiCtugO3Pr3R+8Xl8Fz2FBIJBIUKpIafVxr+5XZ5kBERNQ6DM/UYVniGsc1FRXI3bUbWUnbUVNSAo8HByLg9dfg8eBASCSSuvvZYr8yERGRNWB4tkCWGOpsjaWtcawtLUXOjl3I3r4TuvJyeIUORcD8KLiHBDd6f/YrExERiYPh2cJYWqizVZayxnH1nWJkJyUjZ9du6Kuq0GnkCATOj4Rrn97NPo79ykREROJgeLYwlhLqbJ3Yaxxrbt1CVkIS8vbsg76mBt5jRiMgch5cegQZfA5r6lfmpylERGQrGJ4tjNihrqMQq2e4Ki8P6rhE5O8/AEGvh++E8QiImgenbl1Nel0x8dMUIiKyJQzPFoZfBDMPc/cMV2ZlQ62IQ37qYUikUvg9Ognd5s2Bo5+fSa5nSfhpChER2RKGZwsTHRaCL348hxqdUHfMTibhF8GMzFw9w+WZN6BWxKHw6DFI5XJ0mRmGbnMi4ODd2ajXsWT8NIWIiGwJw7MFEoTmb5NxmLJnuOy336GKUeD2yVOQOjqi29wIZPYIxYdH1Cj801GL6vs1dT8yP00hIiJbwvBsYbakZECnr5+WdXrBZj/iTlWq8K/EHJR8n9RicLOGL52VZFyCOlaBIuU5yFxcEPj4AnQJn4Gfr9yxyL5fc/Qjc1k9IiKyJQzPFqYjfcTdmuBmyV86EwQBxRfToY5RoPhiOuzc3RG06En4z5gOO2dnAMCWlOMW0fd7/xuQKk2NyevisnpERGRLGJ4tjCk/4ra0mdvWfJHMEr90JggCipRnoY6JQ+nly5B7eaHH4mfgP20KZI6O9e5rCW+KGnsD0hRj12VNy+oRERE1h+HZwpjqI25LnLltTaBsa/g0xRsGQa/H7ZOnoIpRoPzadTj4eKPXC0vgN3kSpPb2jT7GEvp+G3sD0hT2IxMRETWO4dnCmOojbkucuW1NoGxL+DT2GwZBp0Phz8egjo1DxU0VHLv4o88rL8FnwnhI7Zr/V8kS+n4NnU1mPzIREVHTGJ4tkCk+4raEtoH7tSZQtiV8GusNg76mBgWph6GOi0dVdg6cuwei3xuvwfuR0ZDIZAadwxL6fpt6A+LmLIejg53FtPMQERFZMobnDsIS2gbuVxvQ/pWYhpIKXbPBrS3hs71vGPTV1cjbfwBZ8YnQ5BfApVdPBL+1Cp1GPAyJVGrQOe5/DmKG0qbegCyd8yDDMhERkYEYnjsIS2gbaMyE0EC4IR+hoaEG3bc1Ia+tbxh0Gg3y9uxDVkISqm/fhlv/fui1bAm8QodBIpEYfH1LYwmz30RERNaO4bmD6IjBqbVvGGoqKpC7azeyk7dDW1wCjwcHou/KFfB4cKBVh+Z7iT37TUREZO0YnjuQjhacDH3DUFNWhuwdu5CzfSdqysrgFToUAfOj4B4SLEbZREREZMEYnsmmNfeGofpOMbKTtyN3127oKivRacTDCJgfCbe+fcxcJREREVkLhucOzNI2TTEXza3byEpIQt6evdBrtfAeMxoBUZFw6REkdmlERERk4RieOyhL3DTF1Kry8pGVkIi8ffsh6PXwnTAO3SLnwTmgm9ilERERkZVgeO6gLHHTFFOpzM6GWpGAgtRDgEQC38mTEDAvAo7+/mKXRkRERFaG4bmDssRNU4yt4uZNqGLjUXj0Z0jt7FAT+ghiNIG4+bsM3v+6gOgwrc29USAiIiLTYnhuwYXr5diUstfm+oItcdMUYyn7/RrUsQrcOn4SUkdHdJszG5k9QrEx5VqHalMhIiIi42N4bkaqUoXtp+5AqxMA2FbgstRNU9qj5NJlqGMUKFKehczFGYGPzUeX8JmQu7vhgw/3dpg2FSIiIjIdhudmbEnJqAvOtWwlcNnKpimCIKAk/ReoYhQovnARdm5u6P7UE+gyYzrsXFzq7tcR2lSIiIjI9Biem2HNgcuQZeisedMUQRBw5+w5qGIUKL10GXIvT/RY/DT8p02FzNGxwf1tuU2FiIiIzIfhuRnWGrhseRk6Qa/H7ZOnoYpVoPz3a7D39kavpc/D99FJkDk4NPk4a2lT6ahrbxMREVkLhudmRIeF4Msfz9Vr3bDEwHW/5pahq/25scKZucKeoNOh8OfjUCviUHHjJhz9/dHnlZfgM34cpHJ5i4+3hjYVW37TYy5880FERKbG8NyMCaGBuH79Oo5cqrKq/xg31VZSG8aMFc7MEfb0NTUoOHQYakU8qrJz4BQQgL4rX4XP2EcgkcladS5Lb1PpSGtvmwLffBARkTkwPLdgUE8XPBs1TuwyWqWpdhMARg1npgx7eq0WeT8dQFZ8AjT5BXDp2RP9//AmOo8cAYlU2q5zWypr7rG3BHzzQURE5sDwbABr+yh4eLAvdh2/YfD92xrOTBH2dBoN8vbuQ1Z8Eqpv34Zrv77otWwJvEKHQSKRtPm8xmaK14S19thbCr75ICIic2B4bsGF6+XYecZ6PgpOVaqw/4y6VY9pazgzZtirqahEbspuZCclQ1tcAveBD6Dva6/AY9CDFhWaAdO1B1jLlxotFd98EBGROYjy+feiRYswc+ZMREREICIiAmlpaWKUYZD9aSXNfvnO0jT20XVz2hPOosNC4CCv33fc2vPVlJWh5tARKJe8gBtbvsN1wR1bu03D505jcL7Gy+KCM9DyFzLbakJoIJbPHwwfLydIAPh4OWH5/MEW+SbNEhnj9UhERNQSs888C4KAzMxMHDx4EHZ2lj/xXVzReBC11I+Cm6vLQS7D5IcCcPpSvlHaDdqzgoW2uBjZyTuQszMFuspK6PsPxDZND9y063T3DhY8w2/K9gBL/1KjJbOGFVWIiMj6mT29Xrt2DQCwePFi3LlzBwsWLMBTTz1l7jIM5uEsazRAW+pHwU19dC2VSupmMV804vUaC3vN9QNrbt1GdmIScnfvhV6rRefRo1D6QAj+fs6uQd2W+mUvtgdYLr75ICIiU5MIgiC0fDfjOXfuHH744Qe8++670Gq1iI6Oxpo1a/DII480+ziNRoP09HQzVfk/F66XY/upO/XWepbLJJj1sCcG9XRp5pHiELvepq4/d4AUfVVp0J1LA/R6SB8cCLsxoyD19gYAvPd9033a7z0RYPK6W0PsMSYiIiLzGDhwIBzu24TN7DPPQ4cOxdChQ+tuR0VF4dChQy2G51qNPQnTUqJnz55W81FwaCjQs6d4q4NsStlbL1R6VpdgVFE6ely5Br1MCr9HJyFg3hw4+vvX3UepVMKnidlcHy8nhIaGmqV2Q4k9xm2hVCotbhxtCcfX9DjGpsXxNT2OsekZc4ybm7Q1e3g+c+YMtFotRo0aBeBuD7Sl9z5b20fBYtZb2/fbufoORt2+iAFlmdBJpDjr3g8v/HklHHy8G32cta00YW2vCSIiIjIOs6fW0tJSfPnll9i2bRu0Wi0SEhLw/vvvm7sMUVnbutGt0d++HME3TiO4/CaqJXY45TkApz0HwNmnU5PBGeCXvYiIiMg6mD08T5w4EWlpaZgzZw70ej2eeOKJem0cts5WtxAuvXwFqhgF5vyqRJXUHj97DcJpz2BUyRwNnkFuzWyuLb8BISIiIsslSr/Ea6+9htdee02MS4vOlrYQFgQBJem/QBWjQPGFi7Bzc0P3Jxfi924P4tLBG9AUVcLHBMHWVt+AEBERkeWz7GZjG2QLWwgLgoA7585DFaNAacYlyD090ePZp+E/bQpkTk4IBDDhkf4mu74tvQEhIiIi68LwbGbWvEawoNfj9ukzUMcoUPbb77Dv3Bm9lj4H30cnQ9bKFVDubbtwd5bheagMDr628AaEiIiIrBPDs4nd35s7PNgX+8+orWZVCQAQdDoUHjsBdawCFTduwtHfD71ffhG+E8dDKpe36lypShW+SryI0gpt3bHiCl1d2wXQ8pcGTfUGhH3URERE1BKGZxNqrDd3/xm1UbfINiV9TQ0KDx+BKjYeVdnZcArohr4rV8Bn7BhIZLJWn+/+8biXRqvDV4kXUa3Vt9jLbIpl7dhHTURERIZgeDahpnpzT1/Kx/+tnSpSVS3Ta7XIP3AQ6rgEaPLy4dKzB/qvfhOdRz7cptBcq7HxuNe9s9G1Gutlbm5Zu7bOHrOPmoiIiAzB8GxC1tabq9NokLf3J2QlJKL61m249u2LXkueg9dDoZBIJO0+f1ufd2OPa2xZu/bMHlvb74qIiIjEwfBsQtby5cCaikrk7t6D7MRkaIuL4f7AAPRdsRwegwcZJTTXzgYLzdzHQS6DvVza6OyzoePVntlja/ldERERkbgYnk3I0recrikrR87OXcjevgM1pWXwHDIYAQsi4fHAA/Xu154v0jXX51zLSS7BS/MHA0C7xqs9s8di/K74BUUiIiLrw/BsQpa65bS2uBjZyTuQs2s3dBUV8BoeisD5UXDr36/Bfdv7Rbrm+px9/rv6yM9panz2/Vl4ezm168uU7Zk9Nvfvil9QJCIisk4Mzy24cL0cm1L2tjlQtWbLaVOrvl2ErMQk5O7eC311NTqPHomAqEi49urZ5GPa+0W6pmZ9JWg421u7Gsny+YPbNGbtnT025++KX1AkIiKyTgzPzUhVqrD91B1odXe7da11dlBTUAB1fCLy9u2HoNPBZ9xYBETNg3NgQIuPbe8X6ZqbDTZ2gLTUmf7G8AuKRERE1onhuRlbUjLqgnMta5odrMzJgVqRgLwDB6EXgItuvXElKBRzx41EPwOCM9D+L9I1Nxv82fdnG31MewKkJc30N4dfUCQiIrJOUrELsGTWOjtYcVOFK59/gbMvrUBe6mGc9+iPzd3nYrfvKFyrtMfG2DSkKlUGnSs6LAQO8vprO7e2FWL5/MHw8XKCBHf7nGvbMpoKih0hQLZ3XImIiEgcnHluRltmB8VcQaHs2nWoY+Nw6/gJSB0c0HV2ODZkeuBmWf3l5loze26MVoimZoMtfTUSU7KmFhMiIiL6H4bnZkSHheDLH8/Va91oLtyJtYJC6eUrUMUqUHRaCZmzMwKi5qHr7HDI3d2heiOp0ce0ZvbcVK0Qtef8V2IaSip0HS5AWkuLCREREf0Pw3MzJoQG4vr16zhyqcqg2UFDvgBnzJnp4vRfoIpRoDjtAuzcXNH9yYXoMiMMdq4udfex9N7aCaGBcEM+QkNDxS6FiIiIqEUMzy0Y1NMFz0aNM+i+LfVIG2NmWhAE3DmfBnWMAiW/ZkDu6Ykez0TDf/pUyJwaBmJu/kFERERkPAzPRtTSLG9TM9OfbztXt0lIU0FTEATcPnUG6lgFyq7+BvvOndFr6XPwfXQyZA4OTdbEzT+IiIiIjIfh2YhamuVtamZar//fOtJ/2XYOwP+CpqDT4dbxE1DFxqEi8wYc/HzR++UX4DtxAqRyuUF1cfMPIiIiIuNgeDailmZ5m5qZvpdOL+CrxIsYP6QrCg4dgVoRh8qsbDh164q+r70Cn3FjIZHVX+LMktokrHV5PyIiIiJDMDwbWVOzvKlKFao0NS0+Xibo0CvnCpQvbocmLx/OQd3Rf9Xr6DxqZIPQXHteS2qTsPQvKBIRERG1B8OzGdwfcBtjp6/B4JKrGHHnF7jXVEDetw96PrcYnYaHQiJtei8bS2uT6MhrNxMREZHtY3g2g8YCLgBIJIBcp8WQ4st4+M6vcNVVQeXoi9TAsVj/6RJIJJJGzlafpbVJcPMPIiIismUMzwZob09xY0HWQVeN0OJLGH4nA056DTKd/JHUaRByXLvg1ceGGhScActsk+DmH0RERGSrGJ5bcOF6OXaeaV9P8b0B10lXhYfuZCC0+BIc9Vro+z6AZLu+yNC6w9vLCa+2MpizTYKIiIjIfBieW7A/raTdPcXRYSH4vx+OY0jBRQwtvgK5UIOrbj3Q8/EoTAgfhbHtqI9tEkRERETmw/DcguKKxr/kZ2hPsaagEN2Ve7D02j4IOh0yXHvgUo/hmD1vtNECLtskiIiIiMyD4bkFHs6yRgN0Sz3FlTm5yIpLQP7BVEAQ4DdxAgKi5mJsly6mKZSIiIiITI7huQWTB7tj55kSg3uKK1RqqBXxKDh8BBKZDH5TH0XAvDlw8PExV8lEREREZCIMzy0Y1NMFPXv2bLGnuPx6JlSxCtw6dgJSe3t0DZ+BrnMi4NC5k0HXMdYugZa02yARERGRrWF4NkBzPcWlV65CFaNA0ekzkDk5ISByLrrODofcw8Pg8xtrl0BL222QiIiIyNYwPLdR8S+/Qh2jwJ3zabBzc0X3Jx5Hl5lhsHN1bfW5jLVLoKXtNkhERERkaxieW0EQBBSnXYAqRoGSX36F3MMDQdFPwT9sOuyc274pibF2CbS03QaJiIiIbA3DswEEQUDR6TNQxcSh7OpV2HfuhJ7PL4bf1Echc3Bo9/mNtUugJe42SERERGRLGJ5bIJSVIW3lKpRfvw4HX1/0fmkZfCdNhFQuN9o1jLVLIHcbJCIiIjIthucWCJpqyL08UTPoCfyf2hkFeyvhffqgUVexMNYugdxtkIiIiMi0GJ5bIO3cCQXhT/93RrcaQMNVLIyxPJyxdgnkboNEREREpsPwbIDmVrEAwOXhiIiIiDoIhmcDNLeKRXuXh7t31trV+W4fdVmFtt4/s/2CiIiIyDIwPBuguVUsGjsOoMnj97p/U5PSCm3dz+79Z85mExEREVkGqdgFWIPosBA4yGX1jtWuYiGVShp9TFPH79XYrHVT7m0TISIiIiJxMDwbYEJoIJbPHwwfLydIAPh4OWH5/MGYEBoIvV5o9DFNHb+XsTZBISIiIiLzYNuGgZpaxcKnidYNHwM2Jmmu7aOp+xMRERGReDjz3E7NtXS05bFN4WYnREREROLjzHM7tWdjkvsfa8zVNoyx9jQRERER1cfwbATt2ZjEFJua3L+KB1frICIiIjIOhmcr0ZqZ5PauPU1EREREjWN4tgKtnUlublMXIiIiImo7fmHQCrS0Pfj9mlqVg6t1EBEREbUPw7MVaO1McntWACEiIiKiprFtw8Rqe5ULiiohlUqg1wvwaeXqF81tD96Y9qwA0hZc2YOIiIg6CoZnE7q/V7l218HWrn4RHRaCv2w7B909uxbKpJJmZ5JNsYpHY7iyBxEREXUkbNswocZ6lWs117PcGImk+dtiaW0/NhEREZE1Y3g2oZZWtzB09YstKRmo0Qn1jtXoBIsIqFzZg4iIiDoShmcTaml1CwHA4g/3IlWpavZ+lhxQubIHERERdSQMzybU2KoX96vtEW4uQLcloKYqVVj84V7MfiPJoIDeVlzZg4iIiDoShmcTqV2BQqPVQSq926Bc+//3a6lHuLUBtfZLfAVFlRBgWEBvqwmhgVg+fzB8vJwgAeDj5YTl8wfzy4JERERkk7jaRisYuiTb3xXnsev4jbrber0AB7kMy+cPxmffn4XQ4BHNt2C0duk5c2/Pba6VPYiIiIjExvBsIEOXZEtVquoF51q14bWpNZtr+5+bCsWtCaiW3CNNREREZM3YtmEgQ5dka679orCostk+aGO1V/BLfERERESmwfBsIENnc5ub3fX2cqrXI9wYY6yRzC/xEREREZkG2zYMZOgW2U3dD0BdeK1twZj9RlKr+58N0VKPNLfTJiIiImobhmcDRYeF1Ot5rlWlqUGqUlUXPpu634xRQQ0CqqGBvC2a6pHmdtpEREREbcfwbKDaYPlV4kWUVmjrjpdWaOuFz8ZmfYcH++L0pXykvJFUb6a3saBt6vYKc6/EYUk4405ERETtxfDcChNCA7ElJaNeeAYahs97Q7QhM73mDHQddSUOzrgTERGRMYgSnrdv346///3vqKmpwdNPP40nn3xSjDLapLXhs6WZXnOvkWzKVhFL1pFn3ImIiMh4zL7aRl5eHj7//HN8//33SExMxI8//ojffvvN3GW0WWuXgbO0md6OuhKHpf0eiIiIyDqZPTwfO3YMI0eOhKenJ5ydnTFt2jTs3r3b3GW0WWvDp6WtudxRt9O2tN8DERERWSeJIAiNrZZmMv/4xz9QUVGBlStXAgBiY2Nx4cIF/PGPf2z2cRqNBunp6eYosUUXrpdjf1oJiit08HCWYfJgdwzq6dLkfbefugOt7n/DLJdJMOthzyYfQ8bH3wMRERG11sCBA+Hg4FDvmNl7nvV6PSQSSd1tQRDq3W5JY0/ClJRKJUJDQ+sdCw0Fno0y7PGhoUDPnlzloTmNjbGxdfTfgznGuCPj+Joex9i0OL6mxzE2PWOOcXOTtmYPz/7+/jhz5kzd7YKCAvj6+pq7DLMy95cCqXH8PRAREVF7mb3nefTo0Th+/Dhu376NyspK7N27F+PGjTN3GURERERErWb2mWc/Pz+sXLkS0dHR0Gq1iIqKwqBBg8xdBhERERFRq4myzvOsWbMwa9YsMS5NRERERNRmZm/bICIiIiKyVgzPREREREQGEqVtw9qlKjvukmdEREREHRnDcyulKlXYGJsGjVYHACgoqsTG2DQAYIAmIiIisnFs22ilLSkZdcG5lkarw5aUDJEqIiIiIiJzYXhupcKiylYdJyIiIiLbwfDcSt5eTq06TkRERES2g+G5laLDQuAgl9U75iCXITosRKSKiIiIiMhc+IXBVqr9UiBX2yAiIiLqeDjz3AYTQgPxf2un4vUnhgEAPvv+LBZ/uBepSpXIlRERERGRKXHmuY24ZB0RERFRx8OZ5zbiknVEREREHQ9nntvIGpas406IRERERMbFmec2svQl62rbSgqKKiHgf20l7MsmIiIiajuG5zay9CXr2FZCREREZHxs22gjS1+yzhraSoiIiIisDcNzO0wIDbSYsHw/by8nFDQSlC2lrYSIiIjIGrFtw0ZZelsJERERkTXizLONsvS2EiIiIiJrxPBswyy5rYSIiIjIGrFtg4iIiIjIQAzPREREREQGYngmIiIiIjIQwzMRERERkYEYnomIiIiIDMTwTERERERkIIZnIiIiIiIDMTwTERERERmI4ZmIiIiIyEAMz0REREREBuL23M1IVarwr8QclHyfBG8vJ0SHhXC7ayIiIqIOjOG5CalKFTbGpkGj1QEACooqsTE2DQAYoImIiIg6KLZtNGFLSkZdcK6l0eqwJSVDpIqIiIiISGwMz00oLKps1XEiIiIisn0Mz03w9nJq1XEiIiIisn0Mz02IDguBg1xW75iDXIbosBCRKiIiIiIisfELg02o/VLgvxLTUFKh42obRERERMTw3JwJoYFwQz5CQ0PFLoWIiIiILADbNoiIiIiIDMTwTERERERkIIZnIiIiIiIDMTwTERERERmI4ZmIiIiIyEAMz0REREREBmJ4JiIiIiIyEMMzEREREZGBGJ6JiIiIiAzE8ExEREREZCCGZyIiIiIiAzE8ExEREREZiOGZiIiIiMhADM9ERERERAayE7sAQwmCAACorq42+7U1Go3Zr9nRcIxNj2NsWhxf0+MYmxbH1/Q4xqZnrDGuzZu1+fNeEqGxoxaotLQUV65cEbsMIiIiIuog+vXrBzc3t3rHrCY86/V6lJeXQy6XQyKRiF0OEREREdkoQRCg1Wrh4uICqbR+l7PVhGciIiIiIrHxC4NERERERAZieCYiIiIiMhDDMxERERGRgRieiYiIiIgMxPBMRERERGQghmciIiIiIgMxPBMRERERGYjhmYiIiIjIQAzP/7Vo0SLMnDkTERERiIiIQFpaWr2fZ2RkYN68eZg2bRreeecd1NTUiFSpdYqNja0b24iICISGhuKDDz6od5+NGzdi4sSJdffZunWrSNVal7KyMoSHh0OtVgMAjh07hlmzZmHq1Kn4/PPPG31MdnY2nnzySUyfPh0vvvgiysvLzVmy1bl/jH/88UeEh4dj1qxZWLNmDaqrqxs8JiEhAWPGjKl7PTf1u6CG47tmzRpMnTq1buz27dvX4DF8DbfOvWN86NChen+PR44ciWXLljV4DF/Dhtu4cSNmzpyJmTNn4pNPPgHAv8XG1Nj4ivp3WCBBr9cLY8aMEbRabZP3mTlzpnDu3DlBEARhzZo1wtatW81Une25cuWKMGXKFOHWrVv1ji9btkw4e/asSFVZp/Pnzwvh4eHCAw88IKhUKqGyslIYP368cPPmTUGr1QqLFy8WUlNTGzxu6dKlwo4dOwRBEISNGzcKn3zyiblLtxr3j/G1a9eEKVOmCKWlpYJerxdWr14tfPPNNw0e98EHHwjbt283f8FW5v7xFQRBCA8PF/Ly8pp9HF/DhmtsjGvl5+cLkydPFq5fv97gcXwNG+bnn38WHnvsMUGj0QjV1dVCdHS0sH37dv4tNpLGxvcf//iHqH+HOfMM4Nq1awCAxYsXY/bs2fjuu+/q/TwrKwtVVVUYMmQIAGDevHnYvXu3ucu0Ge+99x5WrlyJTp061Tuenp6Of/zjH5g1axY++OADaDQakSq0HjExMVi3bh18fX0BABcuXEBQUBACAwNhZ2eHWbNmNXitarVanD59GtOmTQPA13NL7h9je3t7rFu3Dq6urpBIJOjXrx+ys7MbPO7ixYtISEjArFmz8Oabb6K4uNjcpVuF+8e3srIS2dnZePvttzFr1ix8+eWX0Ov19R7D13Dr3D/G9/rkk0/w+OOPo0ePHg1+xtewYXx8fPDWW2/B3t4ecrkcvXv3RmZmJv8WG0lj41tdXS3q32GGZwAlJSUYNWoUNm3ahH//+9/Ytm0bfv7557qf5+fnw8fHp+62j48P8vLyxCjV6h07dgxVVVUICwurd7y8vBwhISFYtWoVEhISUFJSgr/97W8iVWk9PvroIzz00EN1t+9/rfr6+jZ4rRYVFcHV1RV2dnYA+Hpuyf1j3K1bNzzyyCMAgNu3b2Pr1q2YPHlyg8f5+PjgpZdeQnJyMrp06dKgTYnuun98CwsLMXLkSHz88ceIiYnBmTNnoFAo6j2Gr+HWuX+Ma2VmZuLUqVOIjo5u9HF8DRumb9++dZNrmZmZSElJgUQi4d9iI2lsfMPDw0X9O8zwDGDo0KH45JNP4Obmhk6dOiEqKgqHDh2q+7ler4dEIqm7LQhCvdtkuG3btuHZZ59tcNzFxQX//Oc/0bt3b9jZ2WHx4sX1fgdkGENeq40d4+u59fLy8vD0008jMjISI0aMaPDzTZs2ITQ0FBKJBM8//zyOHDkiQpXWJzAwEJs2bYKvry+cnJywaNGiBn8L+Bo2jh9//BFPPPEE7O3tG/05X8Otc/XqVSxevBirV69GYGAg/xYb2b3jW/tJiVh/hxmeAZw5cwbHjx+vuy0IQt07QQDw9/dHQUFB3e3CwsJGP/6i5lVXV+P06dOYNGlSg59lZ2fXm126/3dAhrn/tVpQUNDgtdqpUyeUlpZCp9M1eR9q3u+//47HH38cc+fOxcsvv9zg56Wlpfj3v/9dd1sQBMhkMjNWaL0uX76MPXv21N1u7G8BX8PGsX//fsyYMaPRn/E13DpKpRLPPPMM3njjDcydO5d/i43s/vEFxP07zPCMuwP8ySefQKPRoKysDAkJCZgyZUrdz7t16wYHBwcolUoAQFJSEsaNGydWuVbr8uXL6NGjB5ydnRv8zNHREZ9++ilUKhUEQcDWrVvr/Q7IMIMHD8b169dx48YN6HQ67Nixo8FrVS6X46GHHsKuXbsAAImJiXw9t0JZWRmee+45vPrqq1i8eHGj93F2dsa//vWvulV7vvvuO76eDSQIAj7++GMUFxdDq9Xixx9/bDB2fA233+3bt1FVVYXAwMBGf87XsOFycnLw8ssvY8OGDZg5cyYA/i02psbGV+y/w5zaAzBx4kSkpaVhzpw50Ov1eOKJJzB06FAsWbIEK1aswIMPPogNGzZg7dq1KCsrwwMPPNBkjxg1TaVSwd/fv96xe8f4gw8+wIsvvgitVothw4Y12t5BzXNwcMD69evxyiuvQKPRYPz48Zg+fToA4J133sGkSZMwefJkrFu3Dm+99Rb+/ve/o0uXLvjss89Ertx6KBQKFBYW4ptvvsE333wDAJg0aRJeffXVemP8l7/8Be+99x6qqqrQo0ePuuWVqHnBwcFYunQpFi5ciJqaGkydOhXh4eEA+Bo2JrVa3eDvMQC+htvg66+/hkajwfr16+uOPf744/xbbCSNje+MGTNE/TssEQRBMMqZiIiIiIhsHNs2iIiIiIgMxPBMRERERGQghmciIiIiIgMxPBMRERERGYjhmYiIiIjIQAzPRERmplarERISgoiIiLr/zZ49u8E21MYQHx+PZcuWtXi/tWvXIj09HcDdpbSOHTtm9FqIiGwB13kmIhKBo6MjkpKS6m7n5eUhPDwcAwcORHBwsNnrOXbsGB577DEAwEcffWT26xMRWQuGZyIiC+Dn54egoCBkZmZi//792LlzJ2QyGXr27Il3330XPj4+WLRoEQYMGAClUomioiJERERgxYoVUKvVmDVrFs6dOwcADW7XOn/+PD799FNUV1ejoKAAo0ePxscff4zPP/8c+fn5ePPNN/HJJ59gw4YNePLJJzF9+nT89NNP2LhxI/R6PVxcXLBmzRoMGjQIf/3rX5GVlYWCggJkZWXBz88Pn376KbcXJiKbx7YNIiILcO7cOdy8eRO///47jhw5AoVCge3bt6Nv375466236u53/fp1/PDDD0hISMCuXbtw8OBBg6+xZcsWrFixArGxsdi5cycOHDiA9PR0rFy5Er6+vtiwYQMGDx5cd//ff/8d69atw1//+lckJydjxYoVeOmll1BWVgYAOHPmDL744gvs3r0bTk5O2LZtm/EGhIjIQnHmmYhIBFVVVYiIiAAA6HQ6eHl54dNPP0V8fDzmzZsHZ2dnAEB0dDQ2b96M6upqAMBjjz0GuVwOuVyO6dOn4+jRo+jbt69B11y/fj0OHz6MzZs349q1a9BoNKioqGjy/idOnMDIkSMRGBgIABg1ahQ6depU1xv98MMPw9XVFQAwYMAAFBcXt20wiIisCMMzEZEI7u95rqVQKCCRSOpu6/V61NTU1N22s/vfn21BECCVSiGRSCAIQt1xrVbb6DWfeuop9O/fH2PHjkVYWBjS0tLqPe5+er2+Xi2116ytx9HRse74/TUQEdkqtm0QEVmQsWPHIi4urm5G+Ntvv8Xw4cNhb28PAEhOToZer0dxcTFSUlIwadIkuLu7Q6vV4rfffgMA7Ny5s8F5S0pKcPHiRbz55puYOnUqcnNzcfPmTej1egCATCarF9KBuzPNR48ehUqlAgAcP34cOTk59Vo7iIg6Gs48ExFZkKioKOTk5GD+/PnQ6/UICgrChg0b6n5eVVWFqKgolJeX44knnsCoUaMAAKtWrcKSJUvQqVMnTJ8+vcF53d3dsXTpUsydOxfOzs7w8/PDsGHDcOPGDYwaNQpTpkzBqlWr8N5779U9pk+fPli3bh2WL18OnU4HR0dHbN68GW5ubiYfByIiSyUR+DkbEZFVWLRoUd0qGEREJA62bRARERERGYgzz0REREREBuLMMxERERGRgRieiYiIiIgMxPBMRERERGQghmciIiIiIgMxPBMRERERGej/AwwQy9ywLBmxAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 864x576 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "x = np.linspace(df.人口.min(),df.人口.max(),100)#以人口最小值为起点，最大值为终点，创建元素个数为100的等差数列\n",
    "f = g[0,0] + (g[0,1] * x) #f是假设函数H\n",
    "\n",
    "fig, ax = plt.subplots(figsize=(12,8))#以其他关键字参数**fig_kw来创建图\n",
    "#figsize=(a,b):figsize 设置图形的大小,b为图形的宽,b为图形的高,单位为英寸\n",
    "ax.plot(x, f, 'r', label='Prediction')   #设置点的横坐标，纵坐标，用红色线，并且设置Prediction为关键字参数\n",
    "ax.scatter(df.人口, df.利润, label='Traning Data')  #以人口为横坐标，利润为纵坐标并且设置Traning Data为关键字参数\n",
    "ax.legend(loc=2)  #legend为显示图例函数，loc为设置图例显示的位置，loc=2即在左上方\n",
    "ax.set_xlabel('Population')  #设置x轴变量\n",
    "ax.set_ylabel('Profit')  #设置x轴变量\n",
    "ax.set_title('Predicted Profit vs. Population Size') #设置表头\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "由于梯度方程式函数也在每个训练迭代中输出一个代价的向量，所以我们也可以绘制。 请注意，代价总是降低 - 这是凸优化问题的一个例子。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "***\n",
    "# visualize cost data（代价数据可视化）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "ename": "AttributeError",
     "evalue": "module 'seaborn' has no attribute 'tsplot'",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mAttributeError\u001b[0m                            Traceback (most recent call last)",
      "\u001b[1;32m<ipython-input-23-2cbe82300678>\u001b[0m in \u001b[0;36m<module>\u001b[1;34m\u001b[0m\n\u001b[1;32m----> 1\u001b[1;33m \u001b[0max\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0msns\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mtsplot\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mdata\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mcost\u001b[0m\u001b[1;33m,\u001b[0m\u001b[0merr_style\u001b[0m\u001b[1;33m=\u001b[0m\u001b[1;34m'ci_band'\u001b[0m\u001b[1;33m,\u001b[0m\u001b[0minterpolate\u001b[0m\u001b[1;33m=\u001b[0m\u001b[1;32mFalse\u001b[0m\u001b[1;33m,\u001b[0m\u001b[0mci\u001b[0m\u001b[1;33m=\u001b[0m\u001b[1;33m[\u001b[0m\u001b[1;36m10\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;36m50\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;36m90\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m,\u001b[0m\u001b[0mn_boot\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0miters\u001b[0m\u001b[1;33m)\u001b[0m  \u001b[1;31m#函数参数解释见笔记\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m      2\u001b[0m \u001b[0max\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mset_xlabel\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;34m\"inters\"\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m      3\u001b[0m \u001b[0max\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mset_ylabel\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;34m\"cost\"\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m      4\u001b[0m \u001b[0mplt\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mshow\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;31mAttributeError\u001b[0m: module 'seaborn' has no attribute 'tsplot'"
     ]
    }
   ],
   "source": [
    "ax=sns.tsplot(data=cost,err_style='ci_band',interpolate=False,ci=[10,50,90],n_boot=iters)  #函数参数解释见笔记\n",
    "ax.set_xlabel(\"inters\")\n",
    "ax.set_ylabel(\"cost\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "另一种调库方式"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig, ax = plt.subplots(figsize=(12,8)) #以其他关键字参数**fig_kw来创建图\n",
    "#figsize=(a,b):figsize 设置图形的大小,b为图形的宽,b为图形的高,单位为英寸\n",
    "ax.plot(np.arange(iters), cost, 'b') #作图:以迭代次数为x，代价函数值为y,线条颜色为红色\n",
    "ax.set_xlabel('Iterations')  #设置x轴变量\n",
    "ax.set_ylabel('Cost')  #设置y轴变量\n",
    "ax.set_title('Error vs. Training Epoch') #设置表头\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## 第四章 多变量线性"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "### 多维特征"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data2= pd.read_csv('ex1data2.txt',names=['square','bedrooms','price']) \n",
    "data2.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "### 特征缩放\n",
    "公式：\n",
    "$$\n",
    "x_{n}=\\frac{x_{n}-\\mu_{n}}{s_{n}}\n",
    "$$\n",
    "其中un是平均值，sn 是标准差。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data2 = (data2 - data2.mean()) / data2.std()\n",
    "data2.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "现在我们重复第1部分的预处理步骤，并对新数据集运行线性回归程序。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# add ones column\n",
    "data2.insert(0, 'Ones', 1)#在第0列插入表头为“ONE”的列，数值为1\n",
    "\n",
    "# set X (training data) and y (target variable)\n",
    "cols = data2.shape[1]  #获取表格df的列数\n",
    "X2 = data2.iloc[:,0:cols-1]#除最后一列外，取其他列的所有行，即X2为O，面积，卧室数组成的列表\n",
    "y2 = data2.iloc[:,cols-1:cols]#取最后一列的所有行，即y2为利润\n",
    "\n",
    "# convert to matrices and initialize theta\n",
    "X2 = np.matrix(X2.values)  #转换为矩阵\n",
    "y2 = np.matrix(y2.values)  #转换为矩阵\n",
    "theta2 = np.matrix(np.array([0,0,0]))\n",
    "\n",
    "# perform linear regression on the data set\n",
    "g2, cost2 = gradientDescent(X2, y2, theta2, alpha, iters)\n",
    "\n",
    "# get the cost (error) of the model\n",
    "computeCost(X2, y2, g2)   #计算代价函数\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "我们也可以快速查看这一个的训练进程。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig, ax = plt.subplots(figsize=(12,8))#以其他关键字参数**fig_kw来创建图\n",
    "#figsize=(a,b):figsize 设置图形的大小,b为图形的宽,b为图形的高,单位为英寸\n",
    "ax.plot(np.arange(iters), cost2, 'r')\n",
    "ax.set_xlabel('Iterations')\n",
    "ax.set_ylabel('Cost')\n",
    "ax.set_title('Error vs. Training Epoch')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们也可以使用scikit-learn的线性回归函数，而不是从头开始实现这些算法。 我们将scikit-learn的线性回归算法应用于第1部分的数据，并看看它的表现。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn import linear_model  #从sklearn库中引入线性模块 \n",
    "model = linear_model.LinearRegression()  #声明对象为线性回归模型\n",
    "model.fit(X, y)           #拟合X,y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "scikit-learn model的预测表现"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "x = np.array(X[:, 1].A1)\n",
    "f = model.predict(X).flatten() #将model.predict(X)中的数据降为一维，并返回源数据的副本\n",
    "\n",
    "fig, ax = plt.subplots(figsize=(12,8))#以其他关键字参数**fig_kw来创建图\n",
    "#figsize=(a,b):figsize 设置图形的大小,b为图形的宽,b为图形的高,单位为英寸\n",
    "ax.plot(x, f, 'r', label='Prediction')   #设置点的横坐标，纵坐标，用红色线，并且设置Prediction为关键字参数\n",
    "ax.scatter(df.人口, df.利润, label='Traning Data')  #以人口为横坐标，利润为纵坐标并且设置Traning Data为关键字参数\n",
    "ax.legend(loc=2)  #legend为显示图例函数，loc为设置图例显示的位置，loc=2即在左上方\n",
    "ax.set_xlabel('Population')  #设置x轴变量\n",
    "ax.set_ylabel('Profit')  #设置x轴变量\n",
    "ax.set_title('Predicted Profit vs. Population Size') #设置表头\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 学习率"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "base = np.logspace(-5, -1, num=4)  #以10的-5次方为起点，10的-5次方为终点，元素数目为4的等比数列\n",
    "candidate = np.sort(np.concatenate((base, base*3))) #将base和base*3两个数组拼接在一起，并将其排序\n",
    "print(candidate)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 4. normal equation（正规方程）\n",
    "正规方程是通过求解下面的方程来找出使得代价函数最小的参数的：$\\frac{\\partial }{\\partial {{\\theta }_{j}}}J\\left( {{\\theta }_{j}} \\right)=0$ 。\n",
    " 假设我们的训练集特征矩阵为 X（包含了${{x}_{0}}=1$）并且我们的训练集结果为向量 y，则利用正规方程解出向量 $\\theta ={{\\left( {{X}^{T}}X \\right)}^{-1}}{{X}^{T}}y$ 。\n",
    "上标T代表矩阵转置，上标-1 代表矩阵的逆。设矩阵$A={{X}^{T}}X$，则：${{\\left( {{X}^{T}}X \\right)}^{-1}}={{A}^{-1}}$\n",
    "\n",
    "梯度下降与正规方程的比较：\n",
    "\n",
    "梯度下降：需要选择学习率α，需要多次迭代，当特征数量n大时也能较好适用，适用于各种类型的模型\t\n",
    "\n",
    "正规方程：不需要选择学习率α，一次计算得出，需要计算${{\\left( {{X}^{T}}X \\right)}^{-1}}$，如果特征数量n较大则运算代价大，因为矩阵逆的计算时间复杂度为O(n3)，通常来说当n小于10000 时还是可以接受的，只适用于线性模型，不适合逻辑回归模型等其他模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 正规方程\n",
    "def normalEqn(X, y):\n",
    "    theta = np.linalg.inv(X.T@X)@X.T@y#X.T@X等价于X.T.dot(X)  .dot()表示点积，也就是矩阵相乘的意思\n",
    "    return theta"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "final_theta2=normalEqn(X, y)#感觉和批量梯度下降的theta的值有点差距\n",
    "final_theta2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "梯度下降为matrix([[-3.63029144,  1.16636235]])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": true
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
